-- Logs begin at Mon 2022-08-29 21:47:46 UTC, end at Mon 2022-08-29 23:31:30 UTC. -- Aug 29 22:23:02.104433 dv-cinder-854268-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 29 22:23:02.114213 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 29 22:23:02.119405 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 29 22:23:02 2022] *** Aug 29 22:23:02.119405 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 29 22:23:02.119405 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 29 22:23:02.119405 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: nodename: dv-cinder-854268-3-iscsi Aug 29 22:23:02.119405 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: machine: x86_64 Aug 29 22:23:02.119405 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: clock source: unix Aug 29 22:23:02.119405 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: pcre jit disabled Aug 29 22:23:02.119405 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: detected number of CPU cores: 4 Aug 29 22:23:02.119405 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: current working directory: / Aug 29 22:23:02.119405 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: detected binary path: /usr/bin/uwsgi-core Aug 29 22:23:02.119405 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: your processes number limit is 47680 Aug 29 22:23:02.119405 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: your memory page size is 4096 bytes Aug 29 22:23:02.119405 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: detected max file descriptor number: 2048 Aug 29 22:23:02.119405 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: lock engine: pthread robust mutexes Aug 29 22:23:02.119405 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: thunder lock: enabled Aug 29 22:23:02.119405 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 29 22:23:02.119405 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 29 22:23:02.157050 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: Python main interpreter initialized at 0x55f883ecd9c0 Aug 29 22:23:02.157050 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: python threads support enabled Aug 29 22:23:02.157050 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: your server socket listen backlog is limited to 100 connections Aug 29 22:23:02.157050 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: your mercy for graceful operations on workers is 90 seconds Aug 29 22:23:02.157412 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: mapped 403077 bytes (393 KB) for 2 cores Aug 29 22:23:02.157482 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: *** Operational MODE: preforking *** Aug 29 22:23:02.157745 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: *** uWSGI is running in multiple interpreter mode *** Aug 29 22:23:02.157745 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: spawned uWSGI master process (pid: 82153) Aug 29 22:23:02.157961 dv-cinder-854268-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 29 22:23:02.160560 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: spawned uWSGI worker 1 (pid: 82165, cores: 1) Aug 29 22:23:02.160560 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: spawned uWSGI worker 2 (pid: 82166, cores: 1) Aug 29 22:23:02.160560 dv-cinder-854268-3-iscsi devstack@keystone.service[82153]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 29 22:23:06.597856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 22:23:06.597856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: from cryptography.utils import int_from_bytes Aug 29 22:23:06.598652 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 22:23:06.598652 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: from cryptography.utils import int_from_bytes Aug 29 22:23:06.599108 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 22:23:06.599108 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: from cryptography.utils import int_from_bytes Aug 29 22:23:06.599241 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 22:23:06.599241 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: from cryptography.utils import int_from_bytes Aug 29 22:23:06.649664 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 22:23:06.659241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 22:23:06.659241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] command line args: [] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 22:23:06.659241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 22:23:06.659241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ================================================================================ {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 22:23:06.659241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] admin_token = **** {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.659607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 22:23:06.659607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 22:23:06.659732 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] config_dir = [] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.659797 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] command line args: [] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 22:23:06.659855 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.659855 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] config_source = [] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.659960 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 22:23:06.660017 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] debug = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.660085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ================================================================================ {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 22:23:06.660172 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.660503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] admin_token = **** {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.660568 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.660625 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] config_dir = [] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.660699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] insecure_debug = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.660764 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.660764 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] config_source = [] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.660880 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.660944 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] debug = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.661002 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.661066 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.661162 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] list_limit = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.661228 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.661292 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] log_config_append = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.661348 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] insecure_debug = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.661412 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.661412 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] log_dir = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.661519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.661575 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] log_file = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.661636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.661693 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.661759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] list_limit = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.661823 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.661885 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] log_config_append = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.661885 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.661997 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.662055 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] log_dir = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.662118 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.662118 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.662229 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] log_file = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.662286 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.662343 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.662404 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.662469 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.662527 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.662604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.662667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.662724 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.662789 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.662847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.662904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.662959 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.663020 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.663077 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.663144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.663207 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.663269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] notification_format = cadf {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.663327 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.663383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.663445 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.663445 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.663574 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.59/identity {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.663632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.663687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] publish_errors = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.663750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.663813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.663875 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] notification_format = cadf {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.663937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.663937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.664046 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.664197 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.664197 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.664310 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.59/identity {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.664373 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] standard_threads = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.664443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] publish_errors = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.664515 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] strict_password_check = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.664592 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.664668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.664733 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.664796 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] use_eventlog = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.664854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.664909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] use_journal = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.664909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] use_json = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.665013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.665080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] use_stderr = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.665138 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.665194 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] use_syslog = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.665256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] standard_threads = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.665318 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] watch_log_file = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.665374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] strict_password_check = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.665436 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.665494 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.665550 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.665605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] use_eventlog = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.665660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.665715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] use_journal = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.665715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] use_json = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.665897 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.665961 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] use_stderr = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.666018 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.666073 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] use_syslog = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.666128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.666193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] watch_log_file = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 22:23:06.666255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.666313 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.666368 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.666424 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.666478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.666541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.666611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.666668 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.666723 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.666778 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.666838 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.666907 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.666966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.667021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.667076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.667137 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.667194 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.667257 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.667314 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.667370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.667426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.667498 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.667566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.667631 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.667695 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.667752 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.667808 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.667864 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.667927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.667990 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.668047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.668109 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.668456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.668524 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.668594 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.668653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.668716 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.668774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.668830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.668892 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.669025 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.669094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.669152 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.669208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.669270 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.669382 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.669488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.669488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.669687 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.669768 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.669890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.669977 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.670107 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.670184 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.670294 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.670372 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.670473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.670548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.670663 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.670741 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.670857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.670938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.671013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.671077 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.671154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.671154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.671339 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.671436 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.671531 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.671625 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.671727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.671727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.671902 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.671902 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.672106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.672243 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.672353 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.672451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.672554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.672633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.672751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.672833 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.672958 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.673035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.673152 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.673227 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.673353 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.673442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.673522 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.673641 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.673728 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.673728 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.673858 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.673929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.674047 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.674131 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.674203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.674319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.674394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.674509 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.674591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.674671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.674744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.674860 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.674935 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.675056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.675128 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.675247 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.675327 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.675445 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.675529 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.675608 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.675688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.675815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.675898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.675971 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.676042 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.676230 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.676316 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.676467 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.676556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.676705 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.676785 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.676944 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.677024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.677196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.677305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.677502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.677628 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.677735 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.677833 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.677942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.678050 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.678177 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.678290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.678412 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.678528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.678684 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.enabled = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.678801 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.678882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.679040 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.679119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.679291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.679379 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.679537 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.679617 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.679776 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.679855 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.680008 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.680100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.680265 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.680348 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.680498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.680578 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.enabled = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.680732 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.680818 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.680966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.681048 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.681232 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.681316 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.681474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.681554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.681718 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.681804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.681957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.682035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.682189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.682268 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.682430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.682512 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.682766 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.682963 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.683123 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.683233 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.683341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.683439 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.683536 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.683631 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.683711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.683831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.683923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.684077 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.684178 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.687875 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.687995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.688113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.688361 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.688444 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.688517 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.688588 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.688658 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.688728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.688810 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.688884 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.688955 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.689046 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.689122 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.689192 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.689273 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.689351 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.689428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] auth.external = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.689428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] auth.mapped = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.689556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.689556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.689688 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.689775 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.689851 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.689948 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.690029 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] auth.password = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.690029 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] auth.token = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.690157 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.690227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.690315 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.690404 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] catalog.caching = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.690477 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.690555 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.690628 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.690698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.690776 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] auth.external = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.690857 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.690938 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] auth.mapped = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.691009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.691078 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.691148 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.691247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.691315 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] credential.caching = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.691374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] auth.password = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.691481 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] credential.driver = sql {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.691575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] auth.token = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.691649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.691729 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.691808 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.691887 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] catalog.caching = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.691974 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.692056 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.692126 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.692231 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.692306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.692376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.692446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.692529 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.692610 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.692689 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.692762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.692832 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] credential.caching = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.692832 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] credential.driver = sql {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.692958 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.693027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.693125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.693201 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.693277 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.693350 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.693425 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.693570 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.693663 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.693740 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.693810 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.693879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.693949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] federation.caching = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.694027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.694099 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.694169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.694265 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] federation.driver = sql {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.694341 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.694438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.694438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.694583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.694643 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.694708 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.694788 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.694879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.694961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.695035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.695104 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.695181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.695307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.695406 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] federation.caching = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.695550 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.695630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.695794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.695872 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] federation.driver = sql {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.696029 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] identity.caching = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.696112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.696300 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.696381 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.696548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.696630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.696783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.696862 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.697018 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.697145 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.697264 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.697364 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] identity.driver = sql {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.697481 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.697560 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.697717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.697797 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.697963 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.698041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.698202 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] identity.caching = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.698279 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.698430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.698508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.698659 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.698736 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.698887 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.698964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.699130 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.699209 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.699418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.699514 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] identity.driver = sql {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.699672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.699790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.699928 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.700016 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.700194 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.700387 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.700468 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.700649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.700748 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.700887 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.700966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.701123 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.701812 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.701935 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.702082 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.702222 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.702356 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.702510 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.702591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.702750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.702840 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.702989 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.703068 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.703237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.703316 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.703486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.703606 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.703760 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.703896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.704024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.704190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.704326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.704454 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.704549 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.704612 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.704762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.704853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.705000 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.705082 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.705243 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.705329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.705479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.password = **** {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.705558 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.705798 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.705884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.706035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.706123 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.706281 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.706360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.706518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.706599 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.706755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.706833 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.707005 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.707092 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.707252 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.707331 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.707483 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.707557 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.707715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.707795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.707950 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.password = **** {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.708043 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.708212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.708301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.708445 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.708524 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.708680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.708762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.708915 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.708994 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.709146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.user = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.709225 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.709384 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.709471 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.709631 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.709710 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.709862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.709940 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.710106 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.710195 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.710343 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.710423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.710587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.710669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.710820 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.710901 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.711049 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.711128 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.711282 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.711362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.711514 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.user = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.711599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.711744 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.711822 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.711973 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.712059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.712241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.712324 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.712489 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.712568 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.712779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.712857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.713026 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.713107 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.713271 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.713350 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.713502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.713580 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.713743 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.713837 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.713978 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.714066 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.714210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.714289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.714441 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.714520 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.714671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.714749 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.714913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.714992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.715157 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.715237 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.715390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.715487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.715643 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] policy.driver = sql {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.715722 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.715870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.715951 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.716114 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.716212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.716371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.716449 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.716648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] receipt.caching = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.716731 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.716939 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.717617 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.717794 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.718014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.718160 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.718247 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.718321 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.718479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.718569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.718717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.718806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] policy.driver = sql {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.718960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] resource.caching = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.719047 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.719205 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.719291 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.719442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.719531 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] resource.driver = sql {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.719676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.719763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] receipt.caching = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.719927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.720014 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.720195 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.720284 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.720432 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] revoke.caching = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.720511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.720687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.720766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.720925 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.721021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.721166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] resource.caching = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.721252 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] role.cache_time = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.721398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] role.caching = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.721475 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.721646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] role.driver = sql {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.721733 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] resource.driver = sql {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.721878 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] role.list_limit = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.721956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.722126 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.722202 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.722371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.722450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.722603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] revoke.caching = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.722681 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.722852 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.722931 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.723097 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.723176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.723346 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] role.cache_time = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.723423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.723590 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.723666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] role.caching = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.723837 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.723941 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] role.driver = sql {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.724067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] role.list_limit = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.724157 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.724333 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.724411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.724579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.724660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.724825 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.724904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.725071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.725149 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.725316 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.725499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.725577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.725733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.725811 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.725982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.726072 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.726229 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.726308 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.726474 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.726553 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.726720 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.726806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.726948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.727026 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.727198 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.727276 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.727476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.727617 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.727697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.727856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.727934 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.728107 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.728273 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.728350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.728420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.728596 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.728675 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.728851 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.728937 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.729087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.729165 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.729364 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.729454 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.729597 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.729684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.729827 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] token.cache_time = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.729906 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.730061 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] token.caching = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.730140 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.730297 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.730384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.730655 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] token.provider = fernet {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.730747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.730821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.730892 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.731030 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.731118 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.731277 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.731356 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.731514 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.731598 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.731758 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.731847 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.731990 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] token.cache_time = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.732069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.732255 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] token.caching = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.732336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] trust.driver = sql {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.732536 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.732644 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.732776 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.732863 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] token.provider = fernet {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.733009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.733097 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.733254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.733350 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.733532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.733610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.733759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.733846 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.734000 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.734080 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.734262 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.734343 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.734572 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] trust.driver = sql {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.734707 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] database.connection = **** {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.734806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.734883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.735015 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.735095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.735250 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.735329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.735487 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.735570 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.735719 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.735799 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.735951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.736029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.736203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.736298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.736438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.736518 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.736666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.736741 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] database.connection = **** {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.736899 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.736978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.737137 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.737216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.737371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.737451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.737618 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.737696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.737849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.737928 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.738089 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.738168 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.738317 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.738397 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.738557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.738643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.738826 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.738937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.739021 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 22:23:06.739096 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.739296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.739537 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.739788 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.740056 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.740330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.740565 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.740808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.744325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 22:23:06.744325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 22:23:06.769646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:06.769872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:06.770094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:06.771359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:06.774008 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:06.774008 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:06.778677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:06.781237 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:06.808278 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:06.808576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:06.809505 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:06.809735 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:06.873759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:06.874132 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:06.932105 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:06.932359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:06.932460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:06.932660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.101768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.101993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.102142 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.102233 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.102378 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.102487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.102602 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.102688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.102809 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.102893 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.103025 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.103240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.134034 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.134181 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.134267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.134419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.134555 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.134662 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.134768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.134876 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.134974 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.135093 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.135188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.135304 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.135386 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.135517 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.146891 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.147024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.162927 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.163101 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.163174 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.163250 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.163417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.163513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.163588 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.163671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.163805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.163888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.164022 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.164105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.164270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.164385 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 22:23:07.181997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.182196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 22:23:07.182374 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.182536 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.182622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 22:23:07.182738 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 22:23:07.182911 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.183023 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.183106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 22:23:07.183290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.183425 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.183630 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.183815 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.183898 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.184082 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.184183 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.193568 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.196328 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.196462 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.196543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.196613 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.196686 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.196759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.196759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.201422 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=82165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.202006 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=82166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.204350 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=82165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.204637 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.204925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.205009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=82166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.205208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.205291 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.205488 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.205579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.205748 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.205830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.206020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.206108 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.206288 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.206377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.206559 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.206671 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.206835 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.206948 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.207108 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.207221 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.207435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.207723 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.230452 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.231465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.231942 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=82165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.232262 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.236449 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=82166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.236449 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.237678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=82165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.237967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.238253 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.238456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=82166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.238746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.239033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.244626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=82165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.244910 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.245688 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=82166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.245976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.249636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=82165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.249922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.250207 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.250491 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.250574 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=82166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.250768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.250850 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.251046 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.251130 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.251317 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.251400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.251586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.251682 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.253840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.254137 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.254426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.269060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.269330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.270514 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.270798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.273231 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=82165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.274786 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=82166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.276498 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.276799 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.278058 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.278359 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.281557 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=82165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.283399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=82166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.284805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=82165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.286588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=82166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.288012 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.288342 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.288632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.288920 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.289211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.289492 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.289721 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.290025 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.290314 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.290634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.290913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.291299 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.305890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=82165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.306181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.306452 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.306716 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.308027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=82166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.311120 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.311120 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.311120 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.313857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.315397 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.315986 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.317034 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.317407 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.317864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.319420 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.319696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.320078 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=82165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.320392 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.320798 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.321086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.321376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.322424 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=82166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.322732 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.323026 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.323310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.323598 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.332698 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=82165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.332992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.333372 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.333658 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.333979 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.334239 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.334544 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.335426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=82166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.335797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.336080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.336441 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.339200 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.339200 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.339200 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.350451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=82165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.353369 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=82165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.353525 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=82166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.356289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.356676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=82166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.358355 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.360423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.361677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=82165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.361871 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.361959 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.362260 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.363453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=82166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.363750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.364044 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.368998 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.370496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=82165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.370590 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.372081 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=82166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.373978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.374278 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.374568 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.374846 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.375365 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.375666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.375957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.376264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.382661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=82165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.382955 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.383240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.384767 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=82166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.384767 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.385039 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.390808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=82165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.391093 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.391374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.391649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.391924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.392220 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.392495 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.392773 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.392857 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=82166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 22:23:07.393053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.393135 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.393339 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.393431 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.393611 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.393692 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.393873 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.393954 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.394136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.394227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.394413 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.394494 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=82165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.394693 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.394974 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.395262 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.395534 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.395842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.396108 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.396411 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=82166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 22:23:07.428902 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f883ecd9c0 pid: 82165 (default app) Aug 29 22:23:07.431789 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f883ecd9c0 pid: 82166 (default app) Aug 29 22:23:07.432278 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fa529-000c-48e8-ab33-d70e3742a4e1 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:07.432571 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fa529-000c-48e8-ab33-d70e3742a4e1 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:07.432846 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fa529-000c-48e8-ab33-d70e3742a4e1 None None] PATH_INFO: `/v3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:07.437150 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1/1] 10.222.0.59 () {50 vars in 791 bytes} [Mon Aug 29 22:23:07 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:15.466897 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f010d-3969-41fe-9b28-fc7019b71b79 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:15.467534 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f010d-3969-41fe-9b28-fc7019b71b79 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:15.467749 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f010d-3969-41fe-9b28-fc7019b71b79 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:15.472349 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1/2] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:23:15.477953 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1737d-9d6f-4a4a-b722-0f27d714c825 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:15.478531 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1737d-9d6f-4a4a-b722-0f27d714c825 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:15.478531 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1737d-9d6f-4a4a-b722-0f27d714c825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:15.551789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: INFO dbcounter [None req-dca1737d-9d6f-4a4a-b722-0f27d714c825 None None] Registered counter for database keystone Aug 29 22:23:15.573744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG oslo_db.sqlalchemy.engines [None req-dca1737d-9d6f-4a4a-b722-0f27d714c825 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=82165) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 22:23:15.581039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writer thread running {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 22:23:15.766816 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG passlib.handlers.bcrypt [None req-dca1737d-9d6f-4a4a-b722-0f27d714c825 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=82165) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 22:23:15.767107 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG passlib.handlers.bcrypt [None req-dca1737d-9d6f-4a4a-b722-0f27d714c825 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=82165) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 22:23:15.826254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-dca1737d-9d6f-4a4a-b722-0f27d714c825 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:15.828200 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-dca1737d-9d6f-4a4a-b722-0f27d714c825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:15.963964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 2/3] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 29 22:23:16.491870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f010d-3969-41fe-9b28-fc7019b71b79 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:16.563495 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: INFO dbcounter [None req-3e6f010d-3969-41fe-9b28-fc7019b71b79 None None] Registered counter for database keystone Aug 29 22:23:16.582689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG oslo_db.sqlalchemy.engines [None req-3e6f010d-3969-41fe-9b28-fc7019b71b79 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=82166) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 22:23:16.589456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writer thread running {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 22:23:16.712337 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f010d-3969-41fe-9b28-fc7019b71b79 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:16.717231 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f304736-2401-4448-870a-595cd6222f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:16.717851 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f304736-2401-4448-870a-595cd6222f3d admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:16.718127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f304736-2401-4448-870a-595cd6222f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:16.718397 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f304736-2401-4448-870a-595cd6222f3d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:16.726987 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f304736-2401-4448-870a-595cd6222f3d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:16.781141 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5f304736-2401-4448-870a-595cd6222f3d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 22:23:16.781412 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f304736-2401-4448-870a-595cd6222f3d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:16.791695 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 2/4] 10.222.0.59 () {58 vars in 1324 bytes} [Mon Aug 29 22:23:16 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 22:23:16.798535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1737d-9d6f-4a4a-b722-0f27d714c825 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:16.813898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1737d-9d6f-4a4a-b722-0f27d714c825 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:16.818611 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c0342-8d82-40b5-9b1f-fe74b5a8325d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:16.819727 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c0342-8d82-40b5-9b1f-fe74b5a8325d admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:16.820033 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c0342-8d82-40b5-9b1f-fe74b5a8325d admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:16.820338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c0342-8d82-40b5-9b1f-fe74b5a8325d admin admin] PATH_INFO: `/v3/projects/d3d78b58a69d4b10868c64742da904fe` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:16.821933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c0342-8d82-40b5-9b1f-fe74b5a8325d admin admin] RBAC: Authorizing `identity:get_project(project_id=d3d78b58a69d4b10868c64742da904fe)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:16.877862 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ce1c0342-8d82-40b5-9b1f-fe74b5a8325d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 22:23:16.878211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c0342-8d82-40b5-9b1f-fe74b5a8325d admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:16.883375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 3/5] 10.222.0.59 () {56 vars in 1207 bytes} [Mon Aug 29 22:23:16 2022] GET /identity/v3/projects/d3d78b58a69d4b10868c64742da904fe => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:18.834519 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbb5ef-92bd-4e4d-8292-d0575294cbcf None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:18.835070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbb5ef-92bd-4e4d-8292-d0575294cbcf None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:18.835070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbb5ef-92bd-4e4d-8292-d0575294cbcf None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:18.839260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 3/6] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:18.846594 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d119f-750e-4313-9df6-c25d49b20e07 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:18.847165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d119f-750e-4313-9df6-c25d49b20e07 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:18.847165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d119f-750e-4313-9df6-c25d49b20e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:18.867512 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-a50d119f-750e-4313-9df6-c25d49b20e07 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:18.869059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a50d119f-750e-4313-9df6-c25d49b20e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:18.877485 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 4/7] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 29 22:23:19.151669 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbb5ef-92bd-4e4d-8292-d0575294cbcf None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:19.165097 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbb5ef-92bd-4e4d-8292-d0575294cbcf None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:19.169991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc388630-f61a-4249-a988-8bd79b8815e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:19.170532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc388630-f61a-4249-a988-8bd79b8815e2 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:19.170813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc388630-f61a-4249-a988-8bd79b8815e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:19.171084 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc388630-f61a-4249-a988-8bd79b8815e2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:19.178886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc388630-f61a-4249-a988-8bd79b8815e2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:19.180806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc388630-f61a-4249-a988-8bd79b8815e2 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:19.190110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 4/8] 10.222.0.59 () {58 vars in 1324 bytes} [Mon Aug 29 22:23:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 22:23:19.196259 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d119f-750e-4313-9df6-c25d49b20e07 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:19.205593 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d119f-750e-4313-9df6-c25d49b20e07 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:19.210530 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beef696-c827-444d-9d5a-c32464130c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:19.210875 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beef696-c827-444d-9d5a-c32464130c2c admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:19.211152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beef696-c827-444d-9d5a-c32464130c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:19.211420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beef696-c827-444d-9d5a-c32464130c2c admin admin] PATH_INFO: `/v3/users/b7cd207626714450952ac5fa63cc9cad` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:19.212755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beef696-c827-444d-9d5a-c32464130c2c admin admin] RBAC: Authorizing `identity:get_user(user_id=b7cd207626714450952ac5fa63cc9cad)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:19.214644 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beef696-c827-444d-9d5a-c32464130c2c admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:19.216903 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 5/9] 10.222.0.59 () {56 vars in 1198 bytes} [Mon Aug 29 22:23:19 2022] GET /identity/v3/users/b7cd207626714450952ac5fa63cc9cad => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:23:21.217868 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fe560-16d9-4b59-801f-183221bd24c8 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:21.218373 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fe560-16d9-4b59-801f-183221bd24c8 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:21.218373 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fe560-16d9-4b59-801f-183221bd24c8 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:21.219659 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 5/10] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:23:21.224891 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23c66d-514f-4f45-8ee9-e31f5bfe4d61 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:21.225331 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23c66d-514f-4f45-8ee9-e31f5bfe4d61 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:21.225471 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23c66d-514f-4f45-8ee9-e31f5bfe4d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:21.245076 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-3a23c66d-514f-4f45-8ee9-e31f5bfe4d61 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:21.246257 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3a23c66d-514f-4f45-8ee9-e31f5bfe4d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:21.251803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf631ca-9a9f-4563-b975-594d9697c344 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:21.252075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf631ca-9a9f-4563-b975-594d9697c344 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:21.252381 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf631ca-9a9f-4563-b975-594d9697c344 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:21.253683 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 6/11] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:23:21.258783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f93648-241e-442e-8ee3-d6dc57b1e209 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:21.259060 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f93648-241e-442e-8ee3-d6dc57b1e209 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:21.259330 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f93648-241e-442e-8ee3-d6dc57b1e209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:21.304403 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG passlib.handlers.bcrypt [None req-d4f93648-241e-442e-8ee3-d6dc57b1e209 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=82166) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 22:23:21.304797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG passlib.handlers.bcrypt [None req-d4f93648-241e-442e-8ee3-d6dc57b1e209 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=82166) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 22:23:21.332334 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 6/12] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:21.354394 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-d4f93648-241e-442e-8ee3-d6dc57b1e209 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:21.355979 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d4f93648-241e-442e-8ee3-d6dc57b1e209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:21.458459 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 7/13] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:23:21.605293 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23c66d-514f-4f45-8ee9-e31f5bfe4d61 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:21.630346 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:21.711731 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23c66d-514f-4f45-8ee9-e31f5bfe4d61 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:21.726445 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f93648-241e-442e-8ee3-d6dc57b1e209 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:21.754765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f95e20-d547-4d2e-92b1-46f661569a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:21.755405 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f95e20-d547-4d2e-92b1-46f661569a67 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:21.755683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f95e20-d547-4d2e-92b1-46f661569a67 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:21.755965 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f95e20-d547-4d2e-92b1-46f661569a67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:21.769602 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f95e20-d547-4d2e-92b1-46f661569a67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:21.781045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-07f95e20-d547-4d2e-92b1-46f661569a67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:23:21.781045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:21.781045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:21.781045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:21.781045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:21.781045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:21.781045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:21.781045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:21.781045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:21.781045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:21.781045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:21.781045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:21.781045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:21.781045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:21.781045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:21.781045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:21.782115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:21.782115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:21.782115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:21.782115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:21.782115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:21.782115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:21.782115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:21.782115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:21.782115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:21.782115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:21.782115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:21.782115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:21.782115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:21.782115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:21.782115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:21.782115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:21.782934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:21.782934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:21.782934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:21.782934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:21.782934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:21.782934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:21.782934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:21.782934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:21.782934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:21.782934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:21.782934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:23:21.782934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:23:21.782934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 7/14] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:23:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:23:21.787931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f95e20-d547-4d2e-92b1-46f661569a67 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:21.836748 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f93648-241e-442e-8ee3-d6dc57b1e209 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:21.840305 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:21.881523 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d76645-6cdf-49c7-921b-d18f934bde64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:21.882447 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76645-6cdf-49c7-921b-d18f934bde64 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:21.882728 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76645-6cdf-49c7-921b-d18f934bde64 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:21.882998 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76645-6cdf-49c7-921b-d18f934bde64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:21.888762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:21.909895 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f95e20-d547-4d2e-92b1-46f661569a67 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:21.950061 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc7ef8-1017-4609-b2ca-0e62cc0bd86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:21.950851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc7ef8-1017-4609-b2ca-0e62cc0bd86b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:21.951275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc7ef8-1017-4609-b2ca-0e62cc0bd86b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:21.951636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc7ef8-1017-4609-b2ca-0e62cc0bd86b None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:21.952359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc7ef8-1017-4609-b2ca-0e62cc0bd86b None admin] RBAC: Adding query filter params (name=admin) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:21.952666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc7ef8-1017-4609-b2ca-0e62cc0bd86b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:21.954158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc7ef8-1017-4609-b2ca-0e62cc0bd86b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:21.967109 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 8/15] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:23:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:23:21.969765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d76645-6cdf-49c7-921b-d18f934bde64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:21.971298 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d76645-6cdf-49c7-921b-d18f934bde64 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:21.972844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc7ef8-1017-4609-b2ca-0e62cc0bd86b None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:22.088488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 8/16] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:23:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 22:23:22.089724 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc7ef8-1017-4609-b2ca-0e62cc0bd86b None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:22.096987 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d76645-6cdf-49c7-921b-d18f934bde64 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:22.103294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:22.136221 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f98d5-4406-4b49-9fc7-65ac2a90f4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:22.136772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f98d5-4406-4b49-9fc7-65ac2a90f4ec None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:22.137049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f98d5-4406-4b49-9fc7-65ac2a90f4ec None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:22.137326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f98d5-4406-4b49-9fc7-65ac2a90f4ec None admin] PATH_INFO: `/v3/users/b7cd207626714450952ac5fa63cc9cad` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:22.138871 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f98d5-4406-4b49-9fc7-65ac2a90f4ec None admin] RBAC: Authorizing `identity:get_user(user_id=b7cd207626714450952ac5fa63cc9cad)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:22.141658 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f98d5-4406-4b49-9fc7-65ac2a90f4ec None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:22.149361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 9/17] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:21 2022] GET /identity/v3/users/b7cd207626714450952ac5fa63cc9cad => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:23:22.156921 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f98d5-4406-4b49-9fc7-65ac2a90f4ec None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:22.178115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:22.225128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d76645-6cdf-49c7-921b-d18f934bde64 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:22.264402 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98510455-b74a-4b3b-9e49-fcbf4b7d3a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:22.265367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98510455-b74a-4b3b-9e49-fcbf4b7d3a85 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:22.265740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98510455-b74a-4b3b-9e49-fcbf4b7d3a85 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:22.266016 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98510455-b74a-4b3b-9e49-fcbf4b7d3a85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:22.269752 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f98d5-4406-4b49-9fc7-65ac2a90f4ec None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:22.289486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98510455-b74a-4b3b-9e49-fcbf4b7d3a85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:22.291056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98510455-b74a-4b3b-9e49-fcbf4b7d3a85 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:22.310041 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ffb91-4e88-4d8f-9b0a-dcbdd5e7f655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:22.310864 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ffb91-4e88-4d8f-9b0a-dcbdd5e7f655 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:22.311167 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ffb91-4e88-4d8f-9b0a-dcbdd5e7f655 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:22.311442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ffb91-4e88-4d8f-9b0a-dcbdd5e7f655 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:22.312612 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ffb91-4e88-4d8f-9b0a-dcbdd5e7f655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:22.314058 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ffb91-4e88-4d8f-9b0a-dcbdd5e7f655 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:22.315601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 10/18] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:23:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:23:22.316401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-98510455-b74a-4b3b-9e49-fcbf4b7d3a85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:23:22.316401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:22.316401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:22.316401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:22.316401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:22.316401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:22.316401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:22.316401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:22.316401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:22.316401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:22.316401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:22.316401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:22.316401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:23:22.316401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:23:22.316401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:23:22.316401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:23:22.317446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:22.317446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:22.317446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:22.317446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:22.317446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:22.317446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:22.317446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:22.317446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:22.317446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:22.317446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:22.317446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:22.317446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:22.317446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:22.317446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:22.317446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:22.317446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:22.317446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:23:22.318378 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:23:22.318378 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:23:22.318378 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:23:22.318378 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 9/19] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:23:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:23:22.322636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ffb91-4e88-4d8f-9b0a-dcbdd5e7f655 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:22.323561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98510455-b74a-4b3b-9e49-fcbf4b7d3a85 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:22.394521 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:22.419055 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:22.439896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ffb91-4e88-4d8f-9b0a-dcbdd5e7f655 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:22.447737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98510455-b74a-4b3b-9e49-fcbf4b7d3a85 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:22.481060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65088d12-3a73-49ed-9b13-2289f6a1aa5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:22.481690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65088d12-3a73-49ed-9b13-2289f6a1aa5d None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:22.482025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65088d12-3a73-49ed-9b13-2289f6a1aa5d None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:22.482710 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65088d12-3a73-49ed-9b13-2289f6a1aa5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:22.483340 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65088d12-3a73-49ed-9b13-2289f6a1aa5d None admin] RBAC: Adding query filter params (role.id=a836108447dc439e994611bc74d53d9c, scope.domain.id=default, user.id=b7cd207626714450952ac5fa63cc9cad) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:22.483549 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65088d12-3a73-49ed-9b13-2289f6a1aa5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:22.485081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65088d12-3a73-49ed-9b13-2289f6a1aa5d None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:22.490131 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350f438-832a-4d43-aa1d-43bf50d26f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:22.491116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350f438-832a-4d43-aa1d-43bf50d26f1b None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:22.491430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350f438-832a-4d43-aa1d-43bf50d26f1b None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:22.491704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350f438-832a-4d43-aa1d-43bf50d26f1b None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:22.492282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 11/20] 10.222.0.59 () {56 vars in 1322 bytes} [Mon Aug 29 22:23:22 2022] GET /identity/v3/role_assignments?user.id=b7cd207626714450952ac5fa63cc9cad&scope.domain.id=default&role.id=a836108447dc439e994611bc74d53d9c => generated 233 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:23:22.492435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350f438-832a-4d43-aa1d-43bf50d26f1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:22.492719 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350f438-832a-4d43-aa1d-43bf50d26f1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:22.494182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350f438-832a-4d43-aa1d-43bf50d26f1b None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:22.532254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 10/21] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:23:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:23:24.725988 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82954746-94b8-41b5-adcc-a18f794d3adc None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:24.727011 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82954746-94b8-41b5-adcc-a18f794d3adc None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:24.727496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82954746-94b8-41b5-adcc-a18f794d3adc None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:24.729416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 12/22] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:24.734785 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5539d3-df7c-40f1-8038-69508d96e954 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:24.735223 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5539d3-df7c-40f1-8038-69508d96e954 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:24.735320 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5539d3-df7c-40f1-8038-69508d96e954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:24.761233 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-9c5539d3-df7c-40f1-8038-69508d96e954 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:24.762701 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-9c5539d3-df7c-40f1-8038-69508d96e954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:24.850914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 11/23] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:24.910174 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe890d-2666-47cd-ac59-968c64404fd1 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:24.910780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe890d-2666-47cd-ac59-968c64404fd1 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:24.911271 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe890d-2666-47cd-ac59-968c64404fd1 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:24.912835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 13/24] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:24.921496 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520458a2-621d-48f6-98d0-6697632f51d7 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:24.921789 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520458a2-621d-48f6-98d0-6697632f51d7 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:24.922064 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520458a2-621d-48f6-98d0-6697632f51d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:24.947851 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-520458a2-621d-48f6-98d0-6697632f51d7 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:24.949105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-520458a2-621d-48f6-98d0-6697632f51d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:25.000320 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:25.033736 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 12/25] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:25.280179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe890d-2666-47cd-ac59-968c64404fd1 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:25.333008 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429f706-9565-4e83-854d-e7c17c489c6b None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:25.334106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429f706-9565-4e83-854d-e7c17c489c6b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:25.334614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429f706-9565-4e83-854d-e7c17c489c6b None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:25.336589 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 13/26] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:23:25.342356 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef23bea-714e-4bd2-a9fe-158575c8c0a5 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:25.343245 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef23bea-714e-4bd2-a9fe-158575c8c0a5 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:25.344020 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef23bea-714e-4bd2-a9fe-158575c8c0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:25.366736 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-2ef23bea-714e-4bd2-a9fe-158575c8c0a5 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:25.368496 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-2ef23bea-714e-4bd2-a9fe-158575c8c0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:25.454551 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe890d-2666-47cd-ac59-968c64404fd1 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:25.475209 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 14/27] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:23:25.481128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c67697-80ac-4499-8d69-f693ca609764 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:25.481703 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c67697-80ac-4499-8d69-f693ca609764 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:25.482358 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c67697-80ac-4499-8d69-f693ca609764 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:25.484597 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 15/28] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:25.492960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f2dae-e307-433b-8144-53fc11206632 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:25.493545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f2dae-e307-433b-8144-53fc11206632 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:25.494045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f2dae-e307-433b-8144-53fc11206632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:25.497036 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa59c78-0d73-4c92-9ede-61c2db74582c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:25.498385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa59c78-0d73-4c92-9ede-61c2db74582c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:25.498958 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa59c78-0d73-4c92-9ede-61c2db74582c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:25.499489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa59c78-0d73-4c92-9ede-61c2db74582c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:25.514419 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa59c78-0d73-4c92-9ede-61c2db74582c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:25.516368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa59c78-0d73-4c92-9ede-61c2db74582c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:25.523579 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:25.534838 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-b43f2dae-e307-433b-8144-53fc11206632 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:25.534838 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b43f2dae-e307-433b-8144-53fc11206632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:25.540656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-4fa59c78-0d73-4c92-9ede-61c2db74582c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:23:25.540656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:25.540656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:25.540656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:25.540656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:25.540656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:25.540656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:25.540656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:25.540656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:25.540656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:25.540656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:25.540656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:25.540656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:23:25.540656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:23:25.540656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:23:25.540656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:23:25.541503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:25.541503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:25.541503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:25.541503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:25.541503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:25.541503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:25.541503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:25.541503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:25.541503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:25.541503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:25.541503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:25.541503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:25.541503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:25.541503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:25.541503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:25.541503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:25.541503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:23:25.542296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:23:25.542296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:23:25.542296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:23:25.542296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 14/29] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:23:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:23:25.548279 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa59c78-0d73-4c92-9ede-61c2db74582c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:25.688179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa59c78-0d73-4c92-9ede-61c2db74582c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:25.691509 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 16/30] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:25.692632 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f2dae-e307-433b-8144-53fc11206632 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:25.727520 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fec6b7-e4a3-49b3-8dec-b1a226f5cae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:25.728212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fec6b7-e4a3-49b3-8dec-b1a226f5cae5 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:25.728676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fec6b7-e4a3-49b3-8dec-b1a226f5cae5 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:25.729102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fec6b7-e4a3-49b3-8dec-b1a226f5cae5 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:25.729873 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fec6b7-e4a3-49b3-8dec-b1a226f5cae5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:25.730314 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fec6b7-e4a3-49b3-8dec-b1a226f5cae5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:25.732521 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fec6b7-e4a3-49b3-8dec-b1a226f5cae5 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:25.753428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:25.755271 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 15/31] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:23:25.761060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fec6b7-e4a3-49b3-8dec-b1a226f5cae5 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:25.829116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:25.855169 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f2dae-e307-433b-8144-53fc11206632 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:25.881548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fec6b7-e4a3-49b3-8dec-b1a226f5cae5 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:25.896114 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5504d1-6a42-4eeb-b6cb-173cb303540e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:25.897179 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5504d1-6a42-4eeb-b6cb-173cb303540e None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:25.897672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5504d1-6a42-4eeb-b6cb-173cb303540e None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:25.898169 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5504d1-6a42-4eeb-b6cb-173cb303540e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:25.911428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5504d1-6a42-4eeb-b6cb-173cb303540e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:25.921430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-3f5504d1-6a42-4eeb-b6cb-173cb303540e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:23:25.921430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:25.921430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:25.921430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:25.921430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:25.921430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:25.921430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:25.921430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:25.921430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:25.921430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:25.921430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:25.921430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:25.921430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:25.921430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:25.921430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:25.921430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:25.922290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:25.922290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:25.922290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:25.922290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:25.922290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:25.922290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:25.922290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:25.922290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:25.922290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:25.922290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:25.922290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:25.922290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:25.922290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:25.922290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:25.922290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:25.922290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:25.923033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:25.923033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:25.923033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:25.923033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:25.923033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:25.923033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:25.923033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:25.923033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:25.923033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:25.923033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:25.923033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:23:25.923033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:23:25.925705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 17/32] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:23:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:23:25.935236 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5504d1-6a42-4eeb-b6cb-173cb303540e None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:25.945159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9a697-360c-4148-8c13-a114938b3354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:25.947466 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9a697-360c-4148-8c13-a114938b3354 None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:25.947747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9a697-360c-4148-8c13-a114938b3354 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:25.948019 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9a697-360c-4148-8c13-a114938b3354 None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:25.948709 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9a697-360c-4148-8c13-a114938b3354 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:25.950334 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9a697-360c-4148-8c13-a114938b3354 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:26.066281 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:26.072401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5504d1-6a42-4eeb-b6cb-173cb303540e None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:26.109748 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552c66d-fb7a-4211-81ec-53da20782881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:26.110277 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552c66d-fb7a-4211-81ec-53da20782881 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:26.110615 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552c66d-fb7a-4211-81ec-53da20782881 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:26.110909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552c66d-fb7a-4211-81ec-53da20782881 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:26.111591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552c66d-fb7a-4211-81ec-53da20782881 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:26.111888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552c66d-fb7a-4211-81ec-53da20782881 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:26.113578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552c66d-fb7a-4211-81ec-53da20782881 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:26.128046 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 18/33] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:23:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:26.128892 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552c66d-fb7a-4211-81ec-53da20782881 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:26.220819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 16/34] 10.222.0.59 () {60 vars in 1141 bytes} [Mon Aug 29 22:23:25 2022] POST /identity/v3/projects => generated 301 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:23:26.221802 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9a697-360c-4148-8c13-a114938b3354 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:26.265417 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552c66d-fb7a-4211-81ec-53da20782881 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:26.311186 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3babf-e0ec-4f7e-92d0-1c0b395050d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:26.311884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3babf-e0ec-4f7e-92d0-1c0b395050d0 None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:26.312180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3babf-e0ec-4f7e-92d0-1c0b395050d0 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:26.312456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3babf-e0ec-4f7e-92d0-1c0b395050d0 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:26.314010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3babf-e0ec-4f7e-92d0-1c0b395050d0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:26.315475 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3babf-e0ec-4f7e-92d0-1c0b395050d0 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:26.389634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9a697-360c-4148-8c13-a114938b3354 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:26.401181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:26.428032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb16baa-4809-4d2d-95c0-3c26071a1d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:26.428727 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb16baa-4809-4d2d-95c0-3c26071a1d60 None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:26.429030 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb16baa-4809-4d2d-95c0-3c26071a1d60 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:26.429319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb16baa-4809-4d2d-95c0-3c26071a1d60 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:26.429984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb16baa-4809-4d2d-95c0-3c26071a1d60 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:26.431604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb16baa-4809-4d2d-95c0-3c26071a1d60 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:26.510685 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 17/35] 10.222.0.59 () {60 vars in 1131 bytes} [Mon Aug 29 22:23:26 2022] POST /identity/v3/roles => generated 219 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:23:26.511854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb16baa-4809-4d2d-95c0-3c26071a1d60 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:26.556878 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 19/36] 10.222.0.59 () {60 vars in 1131 bytes} [Mon Aug 29 22:23:26 2022] POST /identity/v3/roles => generated 225 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:23:26.558466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26764ac-b667-43da-adaa-9d49ff663dbf None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:26.558753 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26764ac-b667-43da-adaa-9d49ff663dbf None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:26.559028 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26764ac-b667-43da-adaa-9d49ff663dbf None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:26.569416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 20/37] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:26 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:26.576788 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a15d0-dddd-49d5-b13d-fa797ebab8ab None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:26.577097 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a15d0-dddd-49d5-b13d-fa797ebab8ab None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:26.577434 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a15d0-dddd-49d5-b13d-fa797ebab8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:26.614822 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-894a15d0-dddd-49d5-b13d-fa797ebab8ab None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:26.617218 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-894a15d0-dddd-49d5-b13d-fa797ebab8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:26.648094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:26.673710 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb16baa-4809-4d2d-95c0-3c26071a1d60 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:26.749961 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc686e-ab55-485a-bd26-fd7d302a293b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:26.750766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc686e-ab55-485a-bd26-fd7d302a293b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:26.750936 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc686e-ab55-485a-bd26-fd7d302a293b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:26.751269 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc686e-ab55-485a-bd26-fd7d302a293b None admin] PATH_INFO: `/v3/users/b7cd207626714450952ac5fa63cc9cad` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:26.752672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 21/38] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:23:26.754043 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc686e-ab55-485a-bd26-fd7d302a293b None admin] RBAC: Authorizing `identity:get_user(user_id=b7cd207626714450952ac5fa63cc9cad)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:26.756170 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc686e-ab55-485a-bd26-fd7d302a293b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:26.757878 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 18/39] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:26 2022] GET /identity/v3/users/b7cd207626714450952ac5fa63cc9cad => generated 247 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:26.763246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a15d0-dddd-49d5-b13d-fa797ebab8ab None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:26.839822 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb915528-31f0-4b93-84cf-6c32954c23db None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:26.840167 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb915528-31f0-4b93-84cf-6c32954c23db None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:26.841633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb915528-31f0-4b93-84cf-6c32954c23db None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:26.842717 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 19/40] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:23:26.848691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a9102-34c1-47b8-b7cb-9bf9ab46a1a9 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:26.848913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a9102-34c1-47b8-b7cb-9bf9ab46a1a9 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:26.849137 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a9102-34c1-47b8-b7cb-9bf9ab46a1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:26.883621 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-bf8a9102-34c1-47b8-b7cb-9bf9ab46a1a9 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:26.884843 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a15d0-dddd-49d5-b13d-fa797ebab8ab None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:26.885251 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-bf8a9102-34c1-47b8-b7cb-9bf9ab46a1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:26.930641 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4e9f7-3310-46ed-a927-67687fcaed84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:26.931241 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:26.931366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4e9f7-3310-46ed-a927-67687fcaed84 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:26.931995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4e9f7-3310-46ed-a927-67687fcaed84 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:26.931995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4e9f7-3310-46ed-a927-67687fcaed84 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:26.933272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4e9f7-3310-46ed-a927-67687fcaed84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:26.934894 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4e9f7-3310-46ed-a927-67687fcaed84 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:26.936751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 22/41] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:23:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:26.945144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4e9f7-3310-46ed-a927-67687fcaed84 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:26.965229 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:26.983587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 20/42] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:23:27.056293 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a9102-34c1-47b8-b7cb-9bf9ab46a1a9 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:27.057630 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4e9f7-3310-46ed-a927-67687fcaed84 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:27.094168 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81992fae-f9a0-4946-b915-3db4a0414acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:27.094835 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81992fae-f9a0-4946-b915-3db4a0414acd None admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:27.095120 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81992fae-f9a0-4946-b915-3db4a0414acd None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:27.095396 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81992fae-f9a0-4946-b915-3db4a0414acd None admin] PATH_INFO: `/v3/domains/default/users/b7cd207626714450952ac5fa63cc9cad/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:27.097604 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81992fae-f9a0-4946-b915-3db4a0414acd None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b7cd207626714450952ac5fa63cc9cad, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:27.099064 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81992fae-f9a0-4946-b915-3db4a0414acd None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:27.162631 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a9102-34c1-47b8-b7cb-9bf9ab46a1a9 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:27.198550 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc7920-2e31-4953-986f-02cc6aecb149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:27.199076 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc7920-2e31-4953-986f-02cc6aecb149 None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:27.199356 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc7920-2e31-4953-986f-02cc6aecb149 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:27.199629 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc7920-2e31-4953-986f-02cc6aecb149 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:27.200399 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc7920-2e31-4953-986f-02cc6aecb149 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:27.201830 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc7920-2e31-4953-986f-02cc6aecb149 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:27.210842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 23/43] 10.222.0.59 () {58 vars in 1361 bytes} [Mon Aug 29 22:23:26 2022] PUT /identity/v3/domains/default/users/b7cd207626714450952ac5fa63cc9cad/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:23:27.259103 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81992fae-f9a0-4946-b915-3db4a0414acd None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:27.302863 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 21/44] 10.222.0.59 () {60 vars in 1131 bytes} [Mon Aug 29 22:23:27 2022] POST /identity/v3/roles => generated 223 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:23:27.350304 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:27.381217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81992fae-f9a0-4946-b915-3db4a0414acd None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:27.420320 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e904d0-ac2d-4f07-8f7f-c280badbaaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:27.421001 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e904d0-ac2d-4f07-8f7f-c280badbaaea None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:27.421282 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e904d0-ac2d-4f07-8f7f-c280badbaaea None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:27.421612 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e904d0-ac2d-4f07-8f7f-c280badbaaea None admin] PATH_INFO: `/v3/domains/default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:27.422890 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e904d0-ac2d-4f07-8f7f-c280badbaaea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:27.424398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e904d0-ac2d-4f07-8f7f-c280badbaaea None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:27.428173 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 24/45] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:23:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:23:27.432051 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc7920-2e31-4953-986f-02cc6aecb149 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:27.464087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:27.548710 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc7920-2e31-4953-986f-02cc6aecb149 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:27.692042 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fef4fb-a868-4105-addb-7d738d6a2c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:27.692681 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fef4fb-a868-4105-addb-7d738d6a2c3e None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:27.692987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fef4fb-a868-4105-addb-7d738d6a2c3e None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:27.693264 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fef4fb-a868-4105-addb-7d738d6a2c3e None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:27.693910 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fef4fb-a868-4105-addb-7d738d6a2c3e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:27.695372 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fef4fb-a868-4105-addb-7d738d6a2c3e None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:27.797047 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 22/46] 10.222.0.59 () {60 vars in 1141 bytes} [Mon Aug 29 22:23:27 2022] POST /identity/v3/projects => generated 312 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:23:29.178988 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b734a7e-f587-48ef-be69-aa26372595cd None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:29.179552 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b734a7e-f587-48ef-be69-aa26372595cd None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:29.179552 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b734a7e-f587-48ef-be69-aa26372595cd None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:29.180906 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 25/47] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:29.187946 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25365bcd-e385-4329-849b-c3cd54d30120 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:29.187946 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25365bcd-e385-4329-849b-c3cd54d30120 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:29.187946 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25365bcd-e385-4329-849b-c3cd54d30120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:29.206089 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-25365bcd-e385-4329-849b-c3cd54d30120 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:29.207275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-25365bcd-e385-4329-849b-c3cd54d30120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:29.276015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 23/48] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:29.549200 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b734a7e-f587-48ef-be69-aa26372595cd None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:29.656698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b734a7e-f587-48ef-be69-aa26372595cd None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:29.691187 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112c101-8449-4c7e-a9bd-d7335e57c275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:29.691826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112c101-8449-4c7e-a9bd-d7335e57c275 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:29.692109 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112c101-8449-4c7e-a9bd-d7335e57c275 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:29.692414 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112c101-8449-4c7e-a9bd-d7335e57c275 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:29.703175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2112c101-8449-4c7e-a9bd-d7335e57c275 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:29.708105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-2112c101-8449-4c7e-a9bd-d7335e57c275 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:23:29.708105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:29.708105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:29.708105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:29.708105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:29.708105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:29.708105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:29.708105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:29.708105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:29.708105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:29.708105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:29.708105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:29.708105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:29.708105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:29.708105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:29.708105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:29.708966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:29.708966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:29.708966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:29.708966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:29.708966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:29.708966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:29.708966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:29.708966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:29.708966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:29.708966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:29.708966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:29.708966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:29.708966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:29.708966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:29.708966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:29.708966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:29.709702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:29.709702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:29.709702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:29.709702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:29.709702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:29.709702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:29.709702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:29.709702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:29.709702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:29.709702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:29.709702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:23:29.709702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:23:29.709702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 26/49] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:23:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:23:29.715282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25365bcd-e385-4329-849b-c3cd54d30120 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:29.725989 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:29.756494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415ab5c-3202-4f73-90ca-05c13c0cdc2a None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:29.756785 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415ab5c-3202-4f73-90ca-05c13c0cdc2a None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:29.757066 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415ab5c-3202-4f73-90ca-05c13c0cdc2a None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:29.758454 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 27/50] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:23:29.763714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9efd5-d559-4f8c-970c-b831fe7054ce None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:29.764087 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9efd5-d559-4f8c-970c-b831fe7054ce None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:29.764336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9efd5-d559-4f8c-970c-b831fe7054ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:29.771504 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:29.783825 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-cca9efd5-d559-4f8c-970c-b831fe7054ce None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:29.785106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-cca9efd5-d559-4f8c-970c-b831fe7054ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:29.821853 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25365bcd-e385-4329-849b-c3cd54d30120 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:29.859476 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cbd6d-109b-461b-9cda-f85ffc4e5abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:29.860110 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cbd6d-109b-461b-9cda-f85ffc4e5abc None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:29.860424 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cbd6d-109b-461b-9cda-f85ffc4e5abc None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:29.860702 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cbd6d-109b-461b-9cda-f85ffc4e5abc None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:29.861401 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cbd6d-109b-461b-9cda-f85ffc4e5abc None admin] RBAC: Adding query filter params (name=admin) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:29.861695 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cbd6d-109b-461b-9cda-f85ffc4e5abc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:29.863123 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cbd6d-109b-461b-9cda-f85ffc4e5abc None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:29.873553 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 24/51] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:23:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:29.879574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cbd6d-109b-461b-9cda-f85ffc4e5abc None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:29.957366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:29.960966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 28/52] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:23:29.988478 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cbd6d-109b-461b-9cda-f85ffc4e5abc None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:30.023179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f7ce2-7780-4c60-8314-1863fb807657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:30.023921 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f7ce2-7780-4c60-8314-1863fb807657 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:30.024231 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f7ce2-7780-4c60-8314-1863fb807657 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:30.024507 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f7ce2-7780-4c60-8314-1863fb807657 None admin] PATH_INFO: `/v3/users/b7cd207626714450952ac5fa63cc9cad` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:30.025635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f7ce2-7780-4c60-8314-1863fb807657 None admin] RBAC: Authorizing `identity:get_user(user_id=b7cd207626714450952ac5fa63cc9cad)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:30.027092 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f7ce2-7780-4c60-8314-1863fb807657 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:30.029293 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 25/53] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:29 2022] GET /identity/v3/users/b7cd207626714450952ac5fa63cc9cad => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:23:30.035025 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9efd5-d559-4f8c-970c-b831fe7054ce None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:30.136868 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9efd5-d559-4f8c-970c-b831fe7054ce None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:30.170801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae9225-6b9d-41fd-b3ca-c76697f691c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:30.171431 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae9225-6b9d-41fd-b3ca-c76697f691c4 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:30.171709 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae9225-6b9d-41fd-b3ca-c76697f691c4 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:30.171981 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae9225-6b9d-41fd-b3ca-c76697f691c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:30.173098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae9225-6b9d-41fd-b3ca-c76697f691c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:30.174559 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae9225-6b9d-41fd-b3ca-c76697f691c4 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:30.176158 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 29/54] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:23:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:30.181895 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f7ce2-7780-4c60-8314-1863fb807657 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:30.239054 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae9225-6b9d-41fd-b3ca-c76697f691c4 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:30.242041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:30.285387 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f7ce2-7780-4c60-8314-1863fb807657 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:30.319679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7396d-3c2c-417e-baed-7f4381674775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:30.320319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7396d-3c2c-417e-baed-7f4381674775 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:30.320597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7396d-3c2c-417e-baed-7f4381674775 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:30.320861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7396d-3c2c-417e-baed-7f4381674775 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:30.321576 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7396d-3c2c-417e-baed-7f4381674775 None admin] RBAC: Adding query filter params (role.id=a836108447dc439e994611bc74d53d9c, scope.domain.id=default, user.id=b7cd207626714450952ac5fa63cc9cad) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:30.321877 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7396d-3c2c-417e-baed-7f4381674775 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:30.323304 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7396d-3c2c-417e-baed-7f4381674775 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:30.329785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 26/55] 10.222.0.59 () {56 vars in 1322 bytes} [Mon Aug 29 22:23:30 2022] GET /identity/v3/role_assignments?user.id=b7cd207626714450952ac5fa63cc9cad&scope.domain.id=default&role.id=a836108447dc439e994611bc74d53d9c => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:30.347175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae9225-6b9d-41fd-b3ca-c76697f691c4 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:30.385527 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb2349-1124-432a-b790-433b5244affb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:30.386181 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb2349-1124-432a-b790-433b5244affb None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:30.386461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb2349-1124-432a-b790-433b5244affb None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:30.386734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb2349-1124-432a-b790-433b5244affb None admin] PATH_INFO: `/v3/domains/default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:30.388864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb2349-1124-432a-b790-433b5244affb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:30.390441 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb2349-1124-432a-b790-433b5244affb None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:30.392120 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 30/56] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:23:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:23:30.398031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7396d-3c2c-417e-baed-7f4381674775 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:30.400959 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:30.510769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7396d-3c2c-417e-baed-7f4381674775 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:30.547133 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d9958-d90d-41b1-8181-89e0f5bb100f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:30.547751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d9958-d90d-41b1-8181-89e0f5bb100f None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:30.548031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d9958-d90d-41b1-8181-89e0f5bb100f None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:30.548330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d9958-d90d-41b1-8181-89e0f5bb100f None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:30.548985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d9958-d90d-41b1-8181-89e0f5bb100f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:30.550517 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d9958-d90d-41b1-8181-89e0f5bb100f None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:30.667351 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 27/57] 10.222.0.59 () {60 vars in 1140 bytes} [Mon Aug 29 22:23:30 2022] POST /identity/v3/projects => generated 298 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:23:32.600369 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b81775-6ddb-4bf9-bc9d-4867d74ffe5f None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:32.600907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b81775-6ddb-4bf9-bc9d-4867d74ffe5f None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:32.600907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b81775-6ddb-4bf9-bc9d-4867d74ffe5f None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:32.602267 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 31/58] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:32.607877 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46080ac9-1a4f-4b5b-89c9-ffa353cba69e None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:32.608315 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46080ac9-1a4f-4b5b-89c9-ffa353cba69e None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:32.608554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46080ac9-1a4f-4b5b-89c9-ffa353cba69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:32.626639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-46080ac9-1a4f-4b5b-89c9-ffa353cba69e None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:32.627819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-46080ac9-1a4f-4b5b-89c9-ffa353cba69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:32.677350 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:32.694297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 28/59] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:32.964928 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b81775-6ddb-4bf9-bc9d-4867d74ffe5f None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:33.034670 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:33.065297 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b81775-6ddb-4bf9-bc9d-4867d74ffe5f None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:33.100751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076218c7-dddf-4923-8b7b-b59b6e0e4f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:33.101373 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076218c7-dddf-4923-8b7b-b59b6e0e4f93 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:33.101655 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076218c7-dddf-4923-8b7b-b59b6e0e4f93 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:33.101934 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076218c7-dddf-4923-8b7b-b59b6e0e4f93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:33.103038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076218c7-dddf-4923-8b7b-b59b6e0e4f93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:33.104501 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076218c7-dddf-4923-8b7b-b59b6e0e4f93 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:33.106063 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 32/60] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:23:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:33.111915 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46080ac9-1a4f-4b5b-89c9-ffa353cba69e None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:33.211835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46080ac9-1a4f-4b5b-89c9-ffa353cba69e None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:33.245697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99c00d-3f97-4bbb-9e8b-ff4517bff73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:33.246308 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99c00d-3f97-4bbb-9e8b-ff4517bff73e None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:33.247627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99c00d-3f97-4bbb-9e8b-ff4517bff73e None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:33.248074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99c00d-3f97-4bbb-9e8b-ff4517bff73e None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:33.248597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99c00d-3f97-4bbb-9e8b-ff4517bff73e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:33.250032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99c00d-3f97-4bbb-9e8b-ff4517bff73e None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:33.363892 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 29/61] 10.222.0.59 () {60 vars in 1132 bytes} [Mon Aug 29 22:23:33 2022] POST /identity/v3/users => generated 275 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:23:36.392806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c339c7-5a0f-4fe8-b53b-e576fc8f128a None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:36.392806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c339c7-5a0f-4fe8-b53b-e576fc8f128a None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:36.392806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c339c7-5a0f-4fe8-b53b-e576fc8f128a None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:36.392806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 33/62] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:36.398295 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86331ed1-bc57-493b-bc21-b304d9aa164f None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:36.399084 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86331ed1-bc57-493b-bc21-b304d9aa164f None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:36.399572 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86331ed1-bc57-493b-bc21-b304d9aa164f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:36.425682 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-86331ed1-bc57-493b-bc21-b304d9aa164f None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:36.427204 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-86331ed1-bc57-493b-bc21-b304d9aa164f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:36.477401 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:36.524418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 30/63] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:23:36.698307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e9316-03ab-4c1e-9224-f0790f4c2a8b None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:36.698307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e9316-03ab-4c1e-9224-f0790f4c2a8b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:36.698307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e9316-03ab-4c1e-9224-f0790f4c2a8b None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:36.699040 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 34/64] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:36.704455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9e10f-43f9-48c5-a360-5dc500c1e6cf None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:36.704998 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9e10f-43f9-48c5-a360-5dc500c1e6cf None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:36.705490 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9e10f-43f9-48c5-a360-5dc500c1e6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:36.724996 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-04d9e10f-43f9-48c5-a360-5dc500c1e6cf None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:36.726498 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-04d9e10f-43f9-48c5-a360-5dc500c1e6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:36.769893 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082540e7-8e04-46aa-95ae-056dd831989a None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:36.769893 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082540e7-8e04-46aa-95ae-056dd831989a None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:36.769893 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082540e7-8e04-46aa-95ae-056dd831989a None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:36.769893 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 35/65] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:36.775345 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66823a7b-ee7a-4f98-a46f-c0d0f5f4cd9c None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:36.775627 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66823a7b-ee7a-4f98-a46f-c0d0f5f4cd9c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:36.776115 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66823a7b-ee7a-4f98-a46f-c0d0f5f4cd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:36.801188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 31/66] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:36.813342 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-66823a7b-ee7a-4f98-a46f-c0d0f5f4cd9c None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:36.814600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-66823a7b-ee7a-4f98-a46f-c0d0f5f4cd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:36.911333 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 36/67] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:36.918191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9e10f-43f9-48c5-a360-5dc500c1e6cf None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:37.092082 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:37.098768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9e10f-43f9-48c5-a360-5dc500c1e6cf None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:37.118296 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d707e-001c-47a6-a04c-bc5ac2ef12f8 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:37.118607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d707e-001c-47a6-a04c-bc5ac2ef12f8 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:37.118866 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d707e-001c-47a6-a04c-bc5ac2ef12f8 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:37.120299 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 37/68] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:23:37.128999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d6842c-f418-4001-b102-d0a97bf66f42 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:37.129284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d6842c-f418-4001-b102-d0a97bf66f42 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:37.129567 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d6842c-f418-4001-b102-d0a97bf66f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:37.146858 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b49cb-08d9-4a1a-a3f3-fea5ffd8920d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:37.147507 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b49cb-08d9-4a1a-a3f3-fea5ffd8920d None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:37.147789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b49cb-08d9-4a1a-a3f3-fea5ffd8920d None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:37.150209 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b49cb-08d9-4a1a-a3f3-fea5ffd8920d None admin] PATH_INFO: `/v3/roles/member` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:37.161161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-d1d6842c-f418-4001-b102-d0a97bf66f42 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:37.162994 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d1d6842c-f418-4001-b102-d0a97bf66f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:37.164658 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b49cb-08d9-4a1a-a3f3-fea5ffd8920d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:37.171683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-bc2b49cb-08d9-4a1a-a3f3-fea5ffd8920d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:23:37.171683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:37.171683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:37.171683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:37.171683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:37.171683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:37.171683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:37.171683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:37.171683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:37.171683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:37.171683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:37.171683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:37.171683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:37.171683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:37.171683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:37.171683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:37.172865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:37.172865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:37.172865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:37.172865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:37.172865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:37.172865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:37.172865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:37.172865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:37.172865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:37.172865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:37.172865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:37.172865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:37.172865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:37.172865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:37.172865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:37.172865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:37.173626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:37.173626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:37.173626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:37.173626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:37.173626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:37.173626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:37.173626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:37.173626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:37.173626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:37.173626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:37.173626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:23:37.173626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:23:37.173626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 32/69] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:23:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:23:37.173626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b49cb-08d9-4a1a-a3f3-fea5ffd8920d None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:37.247141 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:37.260161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 38/70] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:37.260161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d6842c-f418-4001-b102-d0a97bf66f42 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:37.298784 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b49cb-08d9-4a1a-a3f3-fea5ffd8920d None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:37.337623 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06aca50-f170-4069-a259-a34e13ba0066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:37.338252 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06aca50-f170-4069-a259-a34e13ba0066 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:37.338533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06aca50-f170-4069-a259-a34e13ba0066 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:37.338804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06aca50-f170-4069-a259-a34e13ba0066 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:37.350800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06aca50-f170-4069-a259-a34e13ba0066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:37.357646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-b06aca50-f170-4069-a259-a34e13ba0066 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:23:37.357646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:37.357646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:37.357646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:37.357646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:37.357646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:37.357646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:37.357646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:37.357646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:37.357646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:37.357646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:37.357646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:37.357646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:37.357646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:37.357646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:37.357646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:37.358486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:37.358486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:37.358486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:37.358486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:37.358486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:37.358486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:37.358486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:37.358486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:37.358486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:37.358486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:37.358486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:37.358486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:37.358486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:37.358486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:37.358486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:37.358486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:37.359218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:37.359218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:37.359218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:37.359218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:37.359218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:37.359218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:37.359218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:37.359218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:37.359218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:37.359218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:37.359218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:23:37.359218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:23:37.359218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 33/71] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:23:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:23:37.360416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fafd3a-b4a6-4fcb-80db-ddf12f417937 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:37.360680 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fafd3a-b4a6-4fcb-80db-ddf12f417937 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:37.360955 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fafd3a-b4a6-4fcb-80db-ddf12f417937 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:37.362306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 34/72] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:37.363487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fafd3a-b4a6-4fcb-80db-ddf12f417937 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:37.380164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d6842c-f418-4001-b102-d0a97bf66f42 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:37.392238 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:37.419650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672f3a1-f67b-4598-a66c-ece6a7b55518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:37.420343 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672f3a1-f67b-4598-a66c-ece6a7b55518 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:37.420803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672f3a1-f67b-4598-a66c-ece6a7b55518 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:37.421080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672f3a1-f67b-4598-a66c-ece6a7b55518 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:37.421751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672f3a1-f67b-4598-a66c-ece6a7b55518 None admin] RBAC: Adding query filter params (name=member) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:37.422045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672f3a1-f67b-4598-a66c-ece6a7b55518 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:37.423416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672f3a1-f67b-4598-a66c-ece6a7b55518 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:37.436555 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 39/73] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:23:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:37.436555 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672f3a1-f67b-4598-a66c-ece6a7b55518 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:37.486587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fafd3a-b4a6-4fcb-80db-ddf12f417937 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:37.502563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:37.525296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b4f71-3390-4670-8dab-0e734e1547e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:37.526165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b4f71-3390-4670-8dab-0e734e1547e9 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:37.526486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b4f71-3390-4670-8dab-0e734e1547e9 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:37.526755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b4f71-3390-4670-8dab-0e734e1547e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:37.539315 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b4f71-3390-4670-8dab-0e734e1547e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:37.544805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-a29b4f71-3390-4670-8dab-0e734e1547e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:23:37.544805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:37.544805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:37.544805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:37.544805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:37.544805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:37.544805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:37.544805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:37.544805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:37.544805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:37.544805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:37.544805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:37.544805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:37.544805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:37.544805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:37.544805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:37.545744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:37.545744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:37.545744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:37.545744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:37.545744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:37.545744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:37.545744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:37.545744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:37.545744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:37.545744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:37.545744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:37.545744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:37.545744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:37.545744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:37.545744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:37.545744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:37.546649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:37.546649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:37.546649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:37.546649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:37.546649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:37.546649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:37.546649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:37.546649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:37.546649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:37.546649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:37.546649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:23:37.546649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:23:37.546649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 35/74] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:23:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:23:37.551836 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672f3a1-f67b-4598-a66c-ece6a7b55518 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:37.552165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4ec29-f1f7-4ed5-a9fc-86f35757e5cc None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:37.552453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4ec29-f1f7-4ed5-a9fc-86f35757e5cc None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:37.552727 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4ec29-f1f7-4ed5-a9fc-86f35757e5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:37.570292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-49f4ec29-f1f7-4ed5-a9fc-86f35757e5cc None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:37.571471 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-49f4ec29-f1f7-4ed5-a9fc-86f35757e5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:37.588387 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d01c6-1c74-4669-b5a0-9169ded525fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:37.589123 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d01c6-1c74-4669-b5a0-9169ded525fa None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:37.589408 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d01c6-1c74-4669-b5a0-9169ded525fa None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:37.589684 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d01c6-1c74-4669-b5a0-9169ded525fa None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:37.591069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d01c6-1c74-4669-b5a0-9169ded525fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:37.591451 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d01c6-1c74-4669-b5a0-9169ded525fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:37.592858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d01c6-1c74-4669-b5a0-9169ded525fa None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:37.601524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 40/75] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:23:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:37.602287 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d01c6-1c74-4669-b5a0-9169ded525fa None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:37.646255 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 36/76] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:37.649054 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4ec29-f1f7-4ed5-a9fc-86f35757e5cc None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:37.661946 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:37.712948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d01c6-1c74-4669-b5a0-9169ded525fa None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:37.737085 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:37.749893 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f99709-dff9-4a82-97f1-102c2b5e4f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:37.750535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f99709-dff9-4a82-97f1-102c2b5e4f18 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:37.750816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f99709-dff9-4a82-97f1-102c2b5e4f18 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:37.751096 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f99709-dff9-4a82-97f1-102c2b5e4f18 None admin] PATH_INFO: `/v3/users/98ba2c8a82264275b8635caae9c9737c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:37.765682 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4ec29-f1f7-4ed5-a9fc-86f35757e5cc None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:37.768678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f99709-dff9-4a82-97f1-102c2b5e4f18 None admin] RBAC: Authorizing `identity:get_user(user_id=98ba2c8a82264275b8635caae9c9737c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:37.770075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f99709-dff9-4a82-97f1-102c2b5e4f18 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:37.776415 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 41/77] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:37 2022] GET /identity/v3/users/98ba2c8a82264275b8635caae9c9737c => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:37.776415 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f99709-dff9-4a82-97f1-102c2b5e4f18 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:37.800909 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37400f90-d0d9-43a7-bf80-7def9e64e5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:37.801625 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37400f90-d0d9-43a7-bf80-7def9e64e5f0 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:37.801921 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37400f90-d0d9-43a7-bf80-7def9e64e5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:37.802205 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37400f90-d0d9-43a7-bf80-7def9e64e5f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:37.813439 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37400f90-d0d9-43a7-bf80-7def9e64e5f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:37.818765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-37400f90-d0d9-43a7-bf80-7def9e64e5f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:23:37.818765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:37.818765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:37.818765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:37.818765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:37.818765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:37.818765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:37.818765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:37.818765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:37.818765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:37.818765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:37.818765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:37.818765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:37.818765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:37.818765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:37.818765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:37.819597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:37.819597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:37.819597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:37.819597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:37.819597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:37.819597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:37.819597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:37.819597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:37.819597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:37.819597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:37.819597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:37.819597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:37.819597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:37.819597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:37.819597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:37.819597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:37.820506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:37.820506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:37.820506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:37.820506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:37.820506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:37.820506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:37.820506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:37.820506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:37.820506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:37.820506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:37.820506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:23:37.820506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:23:37.820506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 37/78] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:23:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:23:37.826229 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37400f90-d0d9-43a7-bf80-7def9e64e5f0 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:37.893034 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f99709-dff9-4a82-97f1-102c2b5e4f18 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:37.901376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:37.935765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de926cd-af11-43fe-92a9-ef067ca2ab46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:37.936516 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37400f90-d0d9-43a7-bf80-7def9e64e5f0 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:37.936622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de926cd-af11-43fe-92a9-ef067ca2ab46 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:37.936738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de926cd-af11-43fe-92a9-ef067ca2ab46 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:37.936925 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de926cd-af11-43fe-92a9-ef067ca2ab46 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:37.937589 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de926cd-af11-43fe-92a9-ef067ca2ab46 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:37.937885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de926cd-af11-43fe-92a9-ef067ca2ab46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:37.939305 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de926cd-af11-43fe-92a9-ef067ca2ab46 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:37.947317 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 42/79] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:23:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:37.948117 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de926cd-af11-43fe-92a9-ef067ca2ab46 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:37.972558 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5bb71-a6a5-491b-a157-3f476da24af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:37.973193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5bb71-a6a5-491b-a157-3f476da24af1 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:37.973472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5bb71-a6a5-491b-a157-3f476da24af1 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:37.973741 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5bb71-a6a5-491b-a157-3f476da24af1 None admin] PATH_INFO: `/v3/users/b7cd207626714450952ac5fa63cc9cad` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:37.974880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff5bb71-a6a5-491b-a157-3f476da24af1 None admin] RBAC: Authorizing `identity:get_user(user_id=b7cd207626714450952ac5fa63cc9cad)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:37.976369 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff5bb71-a6a5-491b-a157-3f476da24af1 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:37.977927 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 38/80] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:37 2022] GET /identity/v3/users/b7cd207626714450952ac5fa63cc9cad => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:37.978684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5bb71-a6a5-491b-a157-3f476da24af1 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:37.998997 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:38.058750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de926cd-af11-43fe-92a9-ef067ca2ab46 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:38.083824 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5bb71-a6a5-491b-a157-3f476da24af1 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:38.093745 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec1f04-e66b-486f-8e4b-3b464facdb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:38.094364 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec1f04-e66b-486f-8e4b-3b464facdb9c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:38.094652 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec1f04-e66b-486f-8e4b-3b464facdb9c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:38.094926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec1f04-e66b-486f-8e4b-3b464facdb9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:38.118253 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89da1d-e9e1-4560-b9fc-c27cd038b2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:38.118855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89da1d-e9e1-4560-b9fc-c27cd038b2b1 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:38.119136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89da1d-e9e1-4560-b9fc-c27cd038b2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:38.119408 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89da1d-e9e1-4560-b9fc-c27cd038b2b1 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:38.120066 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89da1d-e9e1-4560-b9fc-c27cd038b2b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:38.120405 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89da1d-e9e1-4560-b9fc-c27cd038b2b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:38.122066 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89da1d-e9e1-4560-b9fc-c27cd038b2b1 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:38.127221 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:38.131033 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 39/81] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:23:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:38.131831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89da1d-e9e1-4560-b9fc-c27cd038b2b1 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:38.165973 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaec1f04-e66b-486f-8e4b-3b464facdb9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:38.167363 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaec1f04-e66b-486f-8e4b-3b464facdb9c None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:38.210896 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:38.233041 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89da1d-e9e1-4560-b9fc-c27cd038b2b1 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:38.267829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518c5e5-9486-422c-8e80-4c63d537e458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:38.268484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518c5e5-9486-422c-8e80-4c63d537e458 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:38.268760 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518c5e5-9486-422c-8e80-4c63d537e458 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:38.269033 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518c5e5-9486-422c-8e80-4c63d537e458 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:38.270172 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518c5e5-9486-422c-8e80-4c63d537e458 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:38.270762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 43/82] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:23:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:23:38.271619 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec1f04-e66b-486f-8e4b-3b464facdb9c None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:38.271734 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518c5e5-9486-422c-8e80-4c63d537e458 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:38.273442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 40/83] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:23:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:38.277060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518c5e5-9486-422c-8e80-4c63d537e458 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:38.345218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:38.374895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec1f04-e66b-486f-8e4b-3b464facdb9c None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:38.378079 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518c5e5-9486-422c-8e80-4c63d537e458 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:38.409439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74bbb8-962d-47fa-98ed-99a4865b4832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:38.410071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74bbb8-962d-47fa-98ed-99a4865b4832 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:38.410353 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74bbb8-962d-47fa-98ed-99a4865b4832 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:38.410631 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74bbb8-962d-47fa-98ed-99a4865b4832 None admin] PATH_INFO: `/v3/users/98ba2c8a82264275b8635caae9c9737c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:38.411731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74bbb8-962d-47fa-98ed-99a4865b4832 None admin] RBAC: Authorizing `identity:get_user(user_id=98ba2c8a82264275b8635caae9c9737c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:38.413217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74bbb8-962d-47fa-98ed-99a4865b4832 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:38.413378 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a709760-df98-4541-9e0b-04f167661eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:38.414092 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a709760-df98-4541-9e0b-04f167661eb6 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:38.414329 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a709760-df98-4541-9e0b-04f167661eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:38.414604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a709760-df98-4541-9e0b-04f167661eb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:38.414832 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 44/84] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:38 2022] GET /identity/v3/users/98ba2c8a82264275b8635caae9c9737c => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:38.415583 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74bbb8-962d-47fa-98ed-99a4865b4832 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:38.429595 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:38.487840 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a709760-df98-4541-9e0b-04f167661eb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:38.489369 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a709760-df98-4541-9e0b-04f167661eb6 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:38.521089 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74bbb8-962d-47fa-98ed-99a4865b4832 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:38.559625 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4b85e-ed90-40d3-a3b1-4a913df5bd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:38.560503 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4b85e-ed90-40d3-a3b1-4a913df5bd45 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:38.560783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4b85e-ed90-40d3-a3b1-4a913df5bd45 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:38.561099 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4b85e-ed90-40d3-a3b1-4a913df5bd45 None admin] PATH_INFO: `/v3/users/98ba2c8a82264275b8635caae9c9737c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:38.562427 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4b85e-ed90-40d3-a3b1-4a913df5bd45 None admin] RBAC: Authorizing `identity:get_user(user_id=98ba2c8a82264275b8635caae9c9737c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:38.564165 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4b85e-ed90-40d3-a3b1-4a913df5bd45 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:38.566105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 45/85] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:38 2022] GET /identity/v3/users/98ba2c8a82264275b8635caae9c9737c => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:38.566789 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4b85e-ed90-40d3-a3b1-4a913df5bd45 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:38.572775 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:38.615869 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 41/86] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:23:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:23:38.615869 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a709760-df98-4541-9e0b-04f167661eb6 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:38.666070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:38.689886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4b85e-ed90-40d3-a3b1-4a913df5bd45 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:38.726695 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461448ac-9d53-4ff9-975f-034eb3d86302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:38.727345 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461448ac-9d53-4ff9-975f-034eb3d86302 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:38.727453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a709760-df98-4541-9e0b-04f167661eb6 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:38.727627 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461448ac-9d53-4ff9-975f-034eb3d86302 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:38.727942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461448ac-9d53-4ff9-975f-034eb3d86302 None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:38.729138 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461448ac-9d53-4ff9-975f-034eb3d86302 None admin] RBAC: Authorizing `identity:get_project(project_id=09c1bb7fd3204dffae9576cd2681a4a3)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:38.730592 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461448ac-9d53-4ff9-975f-034eb3d86302 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:38.733143 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 46/87] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:23:38 2022] GET /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:38.733931 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461448ac-9d53-4ff9-975f-034eb3d86302 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:38.764355 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a783bf-8454-428d-bae7-18c2341fbf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:38.764956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a783bf-8454-428d-bae7-18c2341fbf36 None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:38.765231 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a783bf-8454-428d-bae7-18c2341fbf36 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:38.765503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a783bf-8454-428d-bae7-18c2341fbf36 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:38.766176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a783bf-8454-428d-bae7-18c2341fbf36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:38.767631 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a783bf-8454-428d-bae7-18c2341fbf36 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:38.841970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461448ac-9d53-4ff9-975f-034eb3d86302 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:38.878636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 42/88] 10.222.0.59 () {60 vars in 1132 bytes} [Mon Aug 29 22:23:38 2022] POST /identity/v3/users => generated 289 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:23:38.879649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a783bf-8454-428d-bae7-18c2341fbf36 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:38.889499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ad9be-2f0d-4f8b-8e7b-258b408d1065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:38.890250 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ad9be-2f0d-4f8b-8e7b-258b408d1065 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:38.890702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ad9be-2f0d-4f8b-8e7b-258b408d1065 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:38.891039 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ad9be-2f0d-4f8b-8e7b-258b408d1065 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:38.909666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:38.924357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:38.977650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7ad9be-2f0d-4f8b-8e7b-258b408d1065 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:38.979477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7ad9be-2f0d-4f8b-8e7b-258b408d1065 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:39.019681 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a783bf-8454-428d-bae7-18c2341fbf36 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:39.069739 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99001e5-75e1-406a-ab01-c8d014f9055b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:39.070379 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99001e5-75e1-406a-ab01-c8d014f9055b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:39.070644 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99001e5-75e1-406a-ab01-c8d014f9055b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:39.072187 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99001e5-75e1-406a-ab01-c8d014f9055b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:39.123062 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 47/89] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:23:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:23:39.124542 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ad9be-2f0d-4f8b-8e7b-258b408d1065 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:39.175334 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99001e5-75e1-406a-ab01-c8d014f9055b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:39.177112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99001e5-75e1-406a-ab01-c8d014f9055b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:39.197934 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:39.211502 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:39.264198 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ad9be-2f0d-4f8b-8e7b-258b408d1065 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:39.304622 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 43/90] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:23:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:23:39.305970 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99001e5-75e1-406a-ab01-c8d014f9055b None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:39.306820 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932e15d-9f59-4767-becf-e41738c519f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:39.307476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932e15d-9f59-4767-becf-e41738c519f4 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:39.307757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932e15d-9f59-4767-becf-e41738c519f4 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:39.308029 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932e15d-9f59-4767-becf-e41738c519f4 None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:39.309655 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6932e15d-9f59-4767-becf-e41738c519f4 None admin] RBAC: Authorizing `identity:get_project(project_id=09c1bb7fd3204dffae9576cd2681a4a3)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:39.311205 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6932e15d-9f59-4767-becf-e41738c519f4 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:39.312953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 48/91] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:23:39 2022] GET /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:39.313740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932e15d-9f59-4767-becf-e41738c519f4 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:39.433776 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932e15d-9f59-4767-becf-e41738c519f4 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:39.442936 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99001e5-75e1-406a-ab01-c8d014f9055b None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:39.450525 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:39.473973 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183c735-06a1-4589-89d0-8c246653e7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:39.474621 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183c735-06a1-4589-89d0-8c246653e7fc None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:39.474949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183c735-06a1-4589-89d0-8c246653e7fc None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:39.475219 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183c735-06a1-4589-89d0-8c246653e7fc None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:39.476567 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183c735-06a1-4589-89d0-8c246653e7fc None admin] RBAC: Authorizing `identity:get_project(project_id=09c1bb7fd3204dffae9576cd2681a4a3)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:39.477721 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:39.479044 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183c735-06a1-4589-89d0-8c246653e7fc None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:39.486595 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 49/92] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:23:39 2022] GET /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:39.487766 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183c735-06a1-4589-89d0-8c246653e7fc None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:39.490589 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36b17b-86ad-4d0c-9cbc-850743de7807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:39.491875 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36b17b-86ad-4d0c-9cbc-850743de7807 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:39.493310 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36b17b-86ad-4d0c-9cbc-850743de7807 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:39.493997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36b17b-86ad-4d0c-9cbc-850743de7807 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:39.495515 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36b17b-86ad-4d0c-9cbc-850743de7807 None admin] RBAC: Adding query filter params (role.id=454bd427dfc04030a4ec2b0e8a1ed237, scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3, user.id=98ba2c8a82264275b8635caae9c9737c) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:39.496025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36b17b-86ad-4d0c-9cbc-850743de7807 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:39.498747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36b17b-86ad-4d0c-9cbc-850743de7807 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:39.507666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 44/93] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:23:39 2022] GET /identity/v3/role_assignments?user.id=98ba2c8a82264275b8635caae9c9737c&scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3&role.id=454bd427dfc04030a4ec2b0e8a1ed237 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:39.509377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36b17b-86ad-4d0c-9cbc-850743de7807 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:39.629442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183c735-06a1-4589-89d0-8c246653e7fc None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:39.670752 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36b17b-86ad-4d0c-9cbc-850743de7807 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:39.688496 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd710a94-53b9-4267-aaa4-0c793ab17a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:39.689579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd710a94-53b9-4267-aaa4-0c793ab17a9d None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:39.691467 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd710a94-53b9-4267-aaa4-0c793ab17a9d None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:39.691972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd710a94-53b9-4267-aaa4-0c793ab17a9d None admin] PATH_INFO: `/v3/projects/544b0c91ae6c421fbd078eb198cd4729` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:39.694708 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd710a94-53b9-4267-aaa4-0c793ab17a9d None admin] RBAC: Authorizing `identity:get_project(project_id=544b0c91ae6c421fbd078eb198cd4729)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:39.698079 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd710a94-53b9-4267-aaa4-0c793ab17a9d None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:39.702099 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 50/94] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:23:39 2022] GET /identity/v3/projects/544b0c91ae6c421fbd078eb198cd4729 => generated 312 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:39.704159 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd710a94-53b9-4267-aaa4-0c793ab17a9d None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:39.734566 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52555e-6843-4086-a139-b4f3b59e0e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:39.735663 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52555e-6843-4086-a139-b4f3b59e0e34 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:39.736163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52555e-6843-4086-a139-b4f3b59e0e34 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:39.736631 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52555e-6843-4086-a139-b4f3b59e0e34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:39.737566 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52555e-6843-4086-a139-b4f3b59e0e34 None admin] RBAC: Adding query filter params (role.id=a836108447dc439e994611bc74d53d9c, scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3, user.id=b7cd207626714450952ac5fa63cc9cad) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:39.738170 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52555e-6843-4086-a139-b4f3b59e0e34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:39.740015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52555e-6843-4086-a139-b4f3b59e0e34 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:39.748505 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 45/95] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:23:39 2022] GET /identity/v3/role_assignments?user.id=b7cd207626714450952ac5fa63cc9cad&scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3&role.id=a836108447dc439e994611bc74d53d9c => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:39.750156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52555e-6843-4086-a139-b4f3b59e0e34 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:39.794477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:39.830937 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:39.867121 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd710a94-53b9-4267-aaa4-0c793ab17a9d None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:39.916392 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d550e-befe-4203-a089-aaed4279fd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:39.916941 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d550e-befe-4203-a089-aaed4279fd0c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:39.917226 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d550e-befe-4203-a089-aaed4279fd0c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:39.917506 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d550e-befe-4203-a089-aaed4279fd0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:39.918310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d550e-befe-4203-a089-aaed4279fd0c None admin] RBAC: Adding query filter params (role.id=733cd39702ab4cbc860e34886dbb165f, scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3, user.id=98ba2c8a82264275b8635caae9c9737c) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:39.918837 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d550e-befe-4203-a089-aaed4279fd0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:39.920490 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d550e-befe-4203-a089-aaed4279fd0c None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:39.937409 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52555e-6843-4086-a139-b4f3b59e0e34 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:39.948567 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 51/96] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:23:39 2022] GET /identity/v3/role_assignments?user.id=98ba2c8a82264275b8635caae9c9737c&scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3&role.id=733cd39702ab4cbc860e34886dbb165f => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:39.987710 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db33a3b-f8a7-45ae-9b59-3f72ece2afaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:39.988374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db33a3b-f8a7-45ae-9b59-3f72ece2afaa None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:39.988662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db33a3b-f8a7-45ae-9b59-3f72ece2afaa None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:39.988939 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db33a3b-f8a7-45ae-9b59-3f72ece2afaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:39.989696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db33a3b-f8a7-45ae-9b59-3f72ece2afaa None admin] RBAC: Adding query filter params (role.id=454bd427dfc04030a4ec2b0e8a1ed237, scope.project.id=544b0c91ae6c421fbd078eb198cd4729, user.id=98ba2c8a82264275b8635caae9c9737c) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:39.990007 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db33a3b-f8a7-45ae-9b59-3f72ece2afaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:39.991582 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db33a3b-f8a7-45ae-9b59-3f72ece2afaa None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:40.000726 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 46/97] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:23:39 2022] GET /identity/v3/role_assignments?user.id=98ba2c8a82264275b8635caae9c9737c&scope.project.id=544b0c91ae6c421fbd078eb198cd4729&role.id=454bd427dfc04030a4ec2b0e8a1ed237 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:41.348070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474e645-9894-4f68-b20f-7a47e05b4250 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:41.348566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474e645-9894-4f68-b20f-7a47e05b4250 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:41.348638 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474e645-9894-4f68-b20f-7a47e05b4250 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:41.350352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 52/98] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:41.355966 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce74e6-8c25-4b38-af3f-963f01472628 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:41.356728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce74e6-8c25-4b38-af3f-963f01472628 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:41.356959 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce74e6-8c25-4b38-af3f-963f01472628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:41.383989 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-37ce74e6-8c25-4b38-af3f-963f01472628 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:41.385266 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-37ce74e6-8c25-4b38-af3f-963f01472628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:41.478729 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 47/99] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:41.514558 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0ebf2-9060-4850-aae5-6de3880cbb93 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:41.514821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0ebf2-9060-4850-aae5-6de3880cbb93 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:41.515111 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0ebf2-9060-4850-aae5-6de3880cbb93 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:41.516940 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 53/100] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:41.522837 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98ba8a-2f9e-429a-b3c1-29be02e2b11f None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:41.523605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98ba8a-2f9e-429a-b3c1-29be02e2b11f None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:41.524343 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98ba8a-2f9e-429a-b3c1-29be02e2b11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:41.553701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-1b98ba8a-2f9e-429a-b3c1-29be02e2b11f None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:41.555591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-1b98ba8a-2f9e-429a-b3c1-29be02e2b11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:41.622639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:41.653386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 48/101] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:41.891130 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0ebf2-9060-4850-aae5-6de3880cbb93 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:42.022168 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0ebf2-9060-4850-aae5-6de3880cbb93 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:42.041989 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:42.068742 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f978d6a-f43c-4b84-b9c4-7a50e5ff90e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:42.069552 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f978d6a-f43c-4b84-b9c4-7a50e5ff90e8 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:42.069857 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f978d6a-f43c-4b84-b9c4-7a50e5ff90e8 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:42.070181 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f978d6a-f43c-4b84-b9c4-7a50e5ff90e8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:42.089055 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f978d6a-f43c-4b84-b9c4-7a50e5ff90e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:42.095418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-4f978d6a-f43c-4b84-b9c4-7a50e5ff90e8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:23:42.095418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:42.095418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:42.095418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:42.095418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:42.095418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:42.095418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:42.095418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:42.095418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:42.095418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:42.095418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:42.095418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:42.095418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:42.095418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:42.095418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:42.095418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:42.096341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:42.096341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:42.096341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:42.096341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:42.096341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:42.096341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:42.096341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:42.096341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:42.096341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:42.096341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:42.096341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:42.096341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:42.096341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:42.096341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:42.096341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:42.096341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:42.097095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:42.097095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:42.097095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:42.097095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:42.097095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:42.097095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:42.097095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:42.097095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:42.097095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:42.097095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:42.097095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:23:42.097095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:23:42.097095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 54/102] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:23:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:23:42.105272 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98ba8a-2f9e-429a-b3c1-29be02e2b11f None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:42.230132 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f978d6a-f43c-4b84-b9c4-7a50e5ff90e8 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:42.268790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98ba8a-2f9e-429a-b3c1-29be02e2b11f None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:42.321337 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e2dce-ae8d-4ced-b006-ebb44794eb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:42.321951 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e2dce-ae8d-4ced-b006-ebb44794eb85 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:42.322225 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e2dce-ae8d-4ced-b006-ebb44794eb85 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:42.322494 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e2dce-ae8d-4ced-b006-ebb44794eb85 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:42.323156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e2dce-ae8d-4ced-b006-ebb44794eb85 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:42.323452 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e2dce-ae8d-4ced-b006-ebb44794eb85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:42.325621 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e2dce-ae8d-4ced-b006-ebb44794eb85 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:42.346473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 49/103] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:23:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:42.351909 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e2dce-ae8d-4ced-b006-ebb44794eb85 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:42.435326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:42.519612 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f978d6a-f43c-4b84-b9c4-7a50e5ff90e8 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:42.584874 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77f513-023f-4924-9ae2-b1d3a903ea02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:42.585876 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77f513-023f-4924-9ae2-b1d3a903ea02 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:42.586361 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77f513-023f-4924-9ae2-b1d3a903ea02 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:42.586833 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77f513-023f-4924-9ae2-b1d3a903ea02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:42.593005 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77f513-023f-4924-9ae2-b1d3a903ea02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:42.594483 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e2dce-ae8d-4ced-b006-ebb44794eb85 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:42.595017 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77f513-023f-4924-9ae2-b1d3a903ea02 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:42.596926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 55/104] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:23:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:23:42.602577 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77f513-023f-4924-9ae2-b1d3a903ea02 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:42.673794 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab63d55-073e-4816-b289-9e0b1427ddc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:42.674477 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab63d55-073e-4816-b289-9e0b1427ddc7 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:42.674753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab63d55-073e-4816-b289-9e0b1427ddc7 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:42.675021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab63d55-073e-4816-b289-9e0b1427ddc7 None admin] PATH_INFO: `/v3/users/cf4fb11822bf402a8fb5b4947411e894` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:42.703706 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab63d55-073e-4816-b289-9e0b1427ddc7 None admin] RBAC: Authorizing `identity:get_user(user_id=cf4fb11822bf402a8fb5b4947411e894)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:42.704956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:42.705258 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab63d55-073e-4816-b289-9e0b1427ddc7 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:42.707108 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 50/105] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:42 2022] GET /identity/v3/users/cf4fb11822bf402a8fb5b4947411e894 => generated 289 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:23:42.712953 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab63d55-073e-4816-b289-9e0b1427ddc7 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:42.826581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77f513-023f-4924-9ae2-b1d3a903ea02 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:42.845999 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:42.858452 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab63d55-073e-4816-b289-9e0b1427ddc7 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:42.888518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98452292-60b4-4165-afae-6042890f41db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:42.889865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98452292-60b4-4165-afae-6042890f41db None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:42.890386 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98452292-60b4-4165-afae-6042890f41db None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:42.890884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98452292-60b4-4165-afae-6042890f41db None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:42.891822 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98452292-60b4-4165-afae-6042890f41db None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:42.894939 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98452292-60b4-4165-afae-6042890f41db None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:42.939554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade9ae0-ebfd-4fe9-8956-f25cc6c2ba15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:42.940619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade9ae0-ebfd-4fe9-8956-f25cc6c2ba15 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:42.941093 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade9ae0-ebfd-4fe9-8956-f25cc6c2ba15 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:42.941552 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade9ae0-ebfd-4fe9-8956-f25cc6c2ba15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:43.028683 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 56/106] 10.222.0.59 () {60 vars in 1141 bytes} [Mon Aug 29 22:23:42 2022] POST /identity/v3/projects => generated 302 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:23:43.030450 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa392886-00d1-457c-af6a-25180b79d550 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:43.032432 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa392886-00d1-457c-af6a-25180b79d550 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:43.033078 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa392886-00d1-457c-af6a-25180b79d550 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:43.035530 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 57/107] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:43.035859 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade9ae0-ebfd-4fe9-8956-f25cc6c2ba15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:43.040339 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade9ae0-ebfd-4fe9-8956-f25cc6c2ba15 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:43.040767 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c9f41-b078-49ba-9c7c-2f5a26c5a3dd None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:43.041047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c9f41-b078-49ba-9c7c-2f5a26c5a3dd None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:43.041318 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c9f41-b078-49ba-9c7c-2f5a26c5a3dd None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:43.043183 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 58/108] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:43.044026 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e89b5c-3e47-4e98-93d2-4093e60281cd None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:43.044388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e89b5c-3e47-4e98-93d2-4093e60281cd None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:43.044676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e89b5c-3e47-4e98-93d2-4093e60281cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:43.101412 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-06e89b5c-3e47-4e98-93d2-4093e60281cd None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:43.102568 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-06e89b5c-3e47-4e98-93d2-4093e60281cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:43.177668 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:43.187931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 51/109] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:23:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 22:23:43.189943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eabedc-c073-499d-bfbe-48fd857b50d8 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:43.191714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eabedc-c073-499d-bfbe-48fd857b50d8 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:43.191714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eabedc-c073-499d-bfbe-48fd857b50d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:43.213377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-f0eabedc-c073-499d-bfbe-48fd857b50d8 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:43.215719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f0eabedc-c073-499d-bfbe-48fd857b50d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:43.218965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 59/110] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:43.224311 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17008f-c85f-497e-8f2b-2d215c9d7351 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:43.224311 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17008f-c85f-497e-8f2b-2d215c9d7351 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:43.224311 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17008f-c85f-497e-8f2b-2d215c9d7351 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:43.224311 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 60/111] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:43.232342 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17008f-c85f-497e-8f2b-2d215c9d7351 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:43.272600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:43.310849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 52/112] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:43.312184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d56cf-e7c1-4456-8596-e1a7d2ca8603 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:43.312458 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d56cf-e7c1-4456-8596-e1a7d2ca8603 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:43.312977 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d56cf-e7c1-4456-8596-e1a7d2ca8603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:43.337275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-b28d56cf-e7c1-4456-8596-e1a7d2ca8603 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:43.338808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b28d56cf-e7c1-4456-8596-e1a7d2ca8603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:43.364213 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17008f-c85f-497e-8f2b-2d215c9d7351 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:43.416961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5463fb1-8275-4a38-b128-c4f343bb7d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:43.417838 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5463fb1-8275-4a38-b128-c4f343bb7d1f None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:43.418685 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5463fb1-8275-4a38-b128-c4f343bb7d1f None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:43.418984 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5463fb1-8275-4a38-b128-c4f343bb7d1f None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:43.420343 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5463fb1-8275-4a38-b128-c4f343bb7d1f None admin] RBAC: Authorizing `identity:get_project(project_id=09c1bb7fd3204dffae9576cd2681a4a3)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:43.422681 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5463fb1-8275-4a38-b128-c4f343bb7d1f None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:43.424695 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 61/113] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:23:43 2022] GET /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:43.426318 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ad06b-4b0f-4b17-a228-a1b41df7c233 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:43.426777 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ad06b-4b0f-4b17-a228-a1b41df7c233 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:43.427060 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ad06b-4b0f-4b17-a228-a1b41df7c233 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:43.428624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 62/114] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:43.437543 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a71f7-9ac6-4107-814b-a2537bce969d None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:43.437839 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a71f7-9ac6-4107-814b-a2537bce969d None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:43.438098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a71f7-9ac6-4107-814b-a2537bce969d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:43.456672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-753a71f7-9ac6-4107-814b-a2537bce969d None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:43.457999 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 53/115] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:43.458421 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-753a71f7-9ac6-4107-814b-a2537bce969d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:43.458860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d56cf-e7c1-4456-8596-e1a7d2ca8603 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:43.565174 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:43.583829 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 63/116] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:43.646955 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d56cf-e7c1-4456-8596-e1a7d2ca8603 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:43.652911 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a71f7-9ac6-4107-814b-a2537bce969d None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:43.705023 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b665ca-7d5f-4e1a-9545-097c367294c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:43.705641 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b665ca-7d5f-4e1a-9545-097c367294c0 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:43.705917 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b665ca-7d5f-4e1a-9545-097c367294c0 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:43.706188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b665ca-7d5f-4e1a-9545-097c367294c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:43.706905 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b665ca-7d5f-4e1a-9545-097c367294c0 None admin] RBAC: Adding query filter params (role.id=d487417153354d90853ba306bd1edb13, scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3, user.id=cf4fb11822bf402a8fb5b4947411e894) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:43.707206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b665ca-7d5f-4e1a-9545-097c367294c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:43.709010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:43.711703 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b665ca-7d5f-4e1a-9545-097c367294c0 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:43.718760 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 54/117] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:23:43 2022] GET /identity/v3/role_assignments?user.id=cf4fb11822bf402a8fb5b4947411e894&scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3&role.id=d487417153354d90853ba306bd1edb13 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:43.722530 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b665ca-7d5f-4e1a-9545-097c367294c0 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:43.878973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b665ca-7d5f-4e1a-9545-097c367294c0 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:43.888635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a71f7-9ac6-4107-814b-a2537bce969d None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:43.909291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:43.920511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a956679-46b7-49f0-8988-78510b066361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:43.921399 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956679-46b7-49f0-8988-78510b066361 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:43.921874 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956679-46b7-49f0-8988-78510b066361 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:43.922369 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956679-46b7-49f0-8988-78510b066361 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:43.935557 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a956679-46b7-49f0-8988-78510b066361 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:43.941474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-8a956679-46b7-49f0-8988-78510b066361 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:23:43.941474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:43.941474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:43.941474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:43.941474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:43.941474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:43.941474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:43.941474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:43.941474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:43.941474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:43.941474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:43.941474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:43.941474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:43.941474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:43.941474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:43.941474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:43.943031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:43.943031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:43.943031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:43.943031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:43.943031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:43.943031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:43.943031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:43.943031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:43.943031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:43.943031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:43.943031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:43.943031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:43.943031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:43.943031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:43.943031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:43.943031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:43.943791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:43.943791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:43.943791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:43.943791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:43.943791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:43.943791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:43.943791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:43.943791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:43.943791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:43.943791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:43.943791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:23:43.943791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:23:43.946406 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 55/118] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:23:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:23:43.947086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a956679-46b7-49f0-8988-78510b066361 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:43.958607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97a84e-9039-4b8d-85ed-06a6c98563a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:43.959276 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97a84e-9039-4b8d-85ed-06a6c98563a5 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:43.959559 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97a84e-9039-4b8d-85ed-06a6c98563a5 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:43.959833 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97a84e-9039-4b8d-85ed-06a6c98563a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:43.984060 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f97a84e-9039-4b8d-85ed-06a6c98563a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:43.989885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-4f97a84e-9039-4b8d-85ed-06a6c98563a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:23:43.989885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:43.989885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:43.989885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:43.989885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:43.989885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:43.989885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:43.989885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:43.989885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:43.989885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:43.989885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:43.989885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:43.989885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:43.989885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:43.989885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:43.989885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:43.990805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:43.990805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:43.990805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:43.990805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:43.990805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:43.990805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:43.990805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:43.990805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:43.990805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:43.990805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:43.990805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:43.990805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:43.990805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:43.990805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:43.990805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:43.990805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:43.991668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:43.991668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:43.991668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:43.991668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:43.991668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:43.991668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:43.991668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:43.991668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:43.991668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:43.991668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:43.991668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:23:43.991668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:23:43.991668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 64/119] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:23:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:23:43.993349 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97a84e-9039-4b8d-85ed-06a6c98563a5 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:44.091941 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a956679-46b7-49f0-8988-78510b066361 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:44.094681 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:44.114568 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97a84e-9039-4b8d-85ed-06a6c98563a5 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:44.136619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5dfa5b-cca5-495f-89ad-632ad1f00202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:44.138697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dfa5b-cca5-495f-89ad-632ad1f00202 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:44.140153 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dfa5b-cca5-495f-89ad-632ad1f00202 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:44.140589 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dfa5b-cca5-495f-89ad-632ad1f00202 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:44.158702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e519050-5861-4489-b4e6-cb7ee9942544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:44.159177 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5dfa5b-cca5-495f-89ad-632ad1f00202 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:44.159648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e519050-5861-4489-b4e6-cb7ee9942544 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:44.159977 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e519050-5861-4489-b4e6-cb7ee9942544 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:44.160377 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e519050-5861-4489-b4e6-cb7ee9942544 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:44.164770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-9b5dfa5b-cca5-495f-89ad-632ad1f00202 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:23:44.164770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:44.164770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:44.164770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:44.164770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:44.164770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:44.164770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:44.164770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:44.164770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:44.164770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:44.164770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:44.164770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:44.164770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:44.164770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:44.164770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:44.164770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:44.165795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:44.165795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:44.165795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:44.165795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:44.165795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:44.165795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:44.165795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:44.165795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:44.165795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:44.165795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:44.165795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:44.165795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:44.165795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:44.165795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:44.165795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:44.165795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:44.166563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:44.166563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:44.166563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:44.166563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:44.166563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:44.166563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:44.166563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:44.166563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:44.166563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:44.166563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:44.166563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:23:44.166563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:23:44.166563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 56/120] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:23:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:23:44.167208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5dfa5b-cca5-495f-89ad-632ad1f00202 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:44.173965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e519050-5861-4489-b4e6-cb7ee9942544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:44.179456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-9e519050-5861-4489-b4e6-cb7ee9942544 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:23:44.179456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:44.179456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:44.179456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:44.179456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:44.179456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:44.179456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:44.179456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:44.179456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:44.179456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:44.179456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:44.179456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:44.179456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:44.179456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:44.179456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:44.179456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:44.181176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:44.181176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:44.181176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:44.181176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:44.181176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:44.181176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:44.181176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:44.181176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:44.181176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:44.181176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:44.181176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:44.181176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:44.181176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:44.181176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:44.181176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:44.181176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:44.183127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:44.183127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:44.183127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:44.183127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:44.183127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:44.183127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:44.183127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:44.183127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:44.183127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:44.183127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:44.183127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:23:44.183127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:23:44.183127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 65/121] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:23:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:23:44.192173 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e519050-5861-4489-b4e6-cb7ee9942544 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:44.216337 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:44.282466 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5dfa5b-cca5-495f-89ad-632ad1f00202 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:44.300498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e519050-5861-4489-b4e6-cb7ee9942544 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:44.319621 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dca02f-a838-43e4-99c4-ebb9d24a85bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:44.320394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dca02f-a838-43e4-99c4-ebb9d24a85bd None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:44.320656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dca02f-a838-43e4-99c4-ebb9d24a85bd None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:44.320929 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dca02f-a838-43e4-99c4-ebb9d24a85bd None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:44.321596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dca02f-a838-43e4-99c4-ebb9d24a85bd None admin] RBAC: Adding query filter params (name=member) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:44.321899 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dca02f-a838-43e4-99c4-ebb9d24a85bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:44.323291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dca02f-a838-43e4-99c4-ebb9d24a85bd None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:44.337995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 57/122] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:23:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:44.337995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dca02f-a838-43e4-99c4-ebb9d24a85bd None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:44.347384 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a4f2e-3e64-439e-a82b-c4c8d1c62ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:44.348018 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a4f2e-3e64-439e-a82b-c4c8d1c62ae8 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:44.348334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a4f2e-3e64-439e-a82b-c4c8d1c62ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:44.348613 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a4f2e-3e64-439e-a82b-c4c8d1c62ae8 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:44.349296 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a4f2e-3e64-439e-a82b-c4c8d1c62ae8 None admin] RBAC: Adding query filter params (name=member) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:44.349628 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a4f2e-3e64-439e-a82b-c4c8d1c62ae8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:44.351110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a4f2e-3e64-439e-a82b-c4c8d1c62ae8 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:44.360401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 66/123] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:23:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:44.361430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a4f2e-3e64-439e-a82b-c4c8d1c62ae8 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:44.365170 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:44.454612 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dca02f-a838-43e4-99c4-ebb9d24a85bd None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:44.456524 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:44.474582 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a4f2e-3e64-439e-a82b-c4c8d1c62ae8 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:44.493602 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df7455-942a-41e2-843c-e2502bd2c3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:44.494993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df7455-942a-41e2-843c-e2502bd2c3ff None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:44.495424 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df7455-942a-41e2-843c-e2502bd2c3ff None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:44.495952 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df7455-942a-41e2-843c-e2502bd2c3ff None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:44.498011 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df7455-942a-41e2-843c-e2502bd2c3ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:44.498479 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df7455-942a-41e2-843c-e2502bd2c3ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:44.500165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df7455-942a-41e2-843c-e2502bd2c3ff None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:44.510278 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 58/124] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:23:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:44.511503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df7455-942a-41e2-843c-e2502bd2c3ff None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:44.512639 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28270909-041a-4e29-8edb-4b4cc46950b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:44.514033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28270909-041a-4e29-8edb-4b4cc46950b0 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:44.514321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28270909-041a-4e29-8edb-4b4cc46950b0 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:44.514597 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28270909-041a-4e29-8edb-4b4cc46950b0 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:44.515298 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28270909-041a-4e29-8edb-4b4cc46950b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:44.515607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28270909-041a-4e29-8edb-4b4cc46950b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:44.517314 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28270909-041a-4e29-8edb-4b4cc46950b0 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:44.528457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 67/125] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:23:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:44.528770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28270909-041a-4e29-8edb-4b4cc46950b0 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:44.601951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:44.630191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df7455-942a-41e2-843c-e2502bd2c3ff None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:44.640599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28270909-041a-4e29-8edb-4b4cc46950b0 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:44.670395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ade95-3ab2-410c-967d-624995db5f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:44.671344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ade95-3ab2-410c-967d-624995db5f13 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:44.671828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ade95-3ab2-410c-967d-624995db5f13 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:44.672544 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ade95-3ab2-410c-967d-624995db5f13 None admin] PATH_INFO: `/v3/users/98ba2c8a82264275b8635caae9c9737c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:44.673948 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ade95-3ab2-410c-967d-624995db5f13 None admin] RBAC: Authorizing `identity:get_user(user_id=98ba2c8a82264275b8635caae9c9737c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:44.675682 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ade95-3ab2-410c-967d-624995db5f13 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:44.677645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 59/126] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:44 2022] GET /identity/v3/users/98ba2c8a82264275b8635caae9c9737c => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:44.678592 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ade95-3ab2-410c-967d-624995db5f13 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:44.682804 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56521deb-ada7-4dee-893f-597f3cf89d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:44.683230 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56521deb-ada7-4dee-893f-597f3cf89d3b None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:44.683522 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56521deb-ada7-4dee-893f-597f3cf89d3b None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:44.684336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56521deb-ada7-4dee-893f-597f3cf89d3b None admin] PATH_INFO: `/v3/users/98ba2c8a82264275b8635caae9c9737c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:44.686018 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56521deb-ada7-4dee-893f-597f3cf89d3b None admin] RBAC: Authorizing `identity:get_user(user_id=98ba2c8a82264275b8635caae9c9737c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:44.687494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56521deb-ada7-4dee-893f-597f3cf89d3b None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:44.689475 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 68/127] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:44 2022] GET /identity/v3/users/98ba2c8a82264275b8635caae9c9737c => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:44.690517 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56521deb-ada7-4dee-893f-597f3cf89d3b None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:44.715796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:44.801766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ade95-3ab2-410c-967d-624995db5f13 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:44.822789 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56521deb-ada7-4dee-893f-597f3cf89d3b None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:44.847186 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66720376-4b04-4391-9145-1221421a2974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:44.847820 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66720376-4b04-4391-9145-1221421a2974 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:44.848104 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66720376-4b04-4391-9145-1221421a2974 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:44.848487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66720376-4b04-4391-9145-1221421a2974 None admin] PATH_INFO: `/v3/users/b7cd207626714450952ac5fa63cc9cad` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:44.849777 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66720376-4b04-4391-9145-1221421a2974 None admin] RBAC: Authorizing `identity:get_user(user_id=b7cd207626714450952ac5fa63cc9cad)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:44.851354 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66720376-4b04-4391-9145-1221421a2974 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:44.853251 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 60/128] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:44 2022] GET /identity/v3/users/b7cd207626714450952ac5fa63cc9cad => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:44.853930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66720376-4b04-4391-9145-1221421a2974 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:44.864445 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ece01a-1fe8-4dbb-a77f-87fc7bd802a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:44.865422 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ece01a-1fe8-4dbb-a77f-87fc7bd802a6 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:44.865910 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ece01a-1fe8-4dbb-a77f-87fc7bd802a6 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:44.866383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ece01a-1fe8-4dbb-a77f-87fc7bd802a6 None admin] PATH_INFO: `/v3/users/98ba2c8a82264275b8635caae9c9737c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:44.867788 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ece01a-1fe8-4dbb-a77f-87fc7bd802a6 None admin] RBAC: Authorizing `identity:get_user(user_id=98ba2c8a82264275b8635caae9c9737c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:44.869509 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ece01a-1fe8-4dbb-a77f-87fc7bd802a6 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:44.872131 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 69/129] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:44 2022] GET /identity/v3/users/98ba2c8a82264275b8635caae9c9737c => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:44.873529 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ece01a-1fe8-4dbb-a77f-87fc7bd802a6 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:44.876584 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:44.968395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:44.972098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66720376-4b04-4391-9145-1221421a2974 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:44.988025 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ece01a-1fe8-4dbb-a77f-87fc7bd802a6 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:45.009774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323f9b3b-b9c4-40e2-9407-b32e824423b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:45.010463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f9b3b-b9c4-40e2-9407-b32e824423b7 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:45.010753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f9b3b-b9c4-40e2-9407-b32e824423b7 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:45.011027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f9b3b-b9c4-40e2-9407-b32e824423b7 None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:45.012796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323f9b3b-b9c4-40e2-9407-b32e824423b7 None admin] RBAC: Authorizing `identity:get_project(project_id=09c1bb7fd3204dffae9576cd2681a4a3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:45.014904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323f9b3b-b9c4-40e2-9407-b32e824423b7 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:45.016688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 61/130] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:23:44 2022] GET /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:45.022327 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323f9b3b-b9c4-40e2-9407-b32e824423b7 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:45.029665 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b10aff-fa57-4f57-9b22-8db93c1a747b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:45.030699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b10aff-fa57-4f57-9b22-8db93c1a747b None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:45.031191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b10aff-fa57-4f57-9b22-8db93c1a747b None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:45.031660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b10aff-fa57-4f57-9b22-8db93c1a747b None admin] PATH_INFO: `/v3/projects/544b0c91ae6c421fbd078eb198cd4729` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:45.033240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b10aff-fa57-4f57-9b22-8db93c1a747b None admin] RBAC: Authorizing `identity:get_project(project_id=544b0c91ae6c421fbd078eb198cd4729)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:45.035030 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b10aff-fa57-4f57-9b22-8db93c1a747b None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:45.037176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 70/131] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:23:44 2022] GET /identity/v3/projects/544b0c91ae6c421fbd078eb198cd4729 => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:45.038241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b10aff-fa57-4f57-9b22-8db93c1a747b None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:45.114612 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:45.143716 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323f9b3b-b9c4-40e2-9407-b32e824423b7 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:45.150169 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b10aff-fa57-4f57-9b22-8db93c1a747b None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:45.184860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152353e7-4817-4fca-922b-438e3096cfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:45.185492 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152353e7-4817-4fca-922b-438e3096cfe4 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:45.185774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152353e7-4817-4fca-922b-438e3096cfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:45.186046 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152353e7-4817-4fca-922b-438e3096cfe4 None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:45.187466 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152353e7-4817-4fca-922b-438e3096cfe4 None admin] RBAC: Authorizing `identity:get_project(project_id=09c1bb7fd3204dffae9576cd2681a4a3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:45.188488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495aa2e-bf89-47e4-a704-fb110120d6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:45.188945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152353e7-4817-4fca-922b-438e3096cfe4 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:45.189560 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495aa2e-bf89-47e4-a704-fb110120d6eb None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:45.190034 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495aa2e-bf89-47e4-a704-fb110120d6eb None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:45.190577 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495aa2e-bf89-47e4-a704-fb110120d6eb None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:45.190980 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 62/132] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:23:45 2022] GET /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:45.191991 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152353e7-4817-4fca-922b-438e3096cfe4 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:45.192517 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495aa2e-bf89-47e4-a704-fb110120d6eb None admin] RBAC: Authorizing `identity:get_project(project_id=09c1bb7fd3204dffae9576cd2681a4a3)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:45.194165 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495aa2e-bf89-47e4-a704-fb110120d6eb None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:45.196450 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 71/133] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:23:45 2022] GET /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:45.197650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495aa2e-bf89-47e4-a704-fb110120d6eb None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:45.233449 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:45.307307 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152353e7-4817-4fca-922b-438e3096cfe4 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:45.309163 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495aa2e-bf89-47e4-a704-fb110120d6eb None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:45.346994 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32342d-a1f5-4ee3-8b8c-ddb186ce618a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:45.347236 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50b77e-3545-4b24-bc12-f4114729851b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:45.347668 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32342d-a1f5-4ee3-8b8c-ddb186ce618a None admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:45.347950 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32342d-a1f5-4ee3-8b8c-ddb186ce618a None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:45.348312 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32342d-a1f5-4ee3-8b8c-ddb186ce618a None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3/users/98ba2c8a82264275b8635caae9c9737c/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:45.348848 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50b77e-3545-4b24-bc12-f4114729851b None admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:45.349320 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50b77e-3545-4b24-bc12-f4114729851b None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:45.349782 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50b77e-3545-4b24-bc12-f4114729851b None admin] PATH_INFO: `/v3/projects/544b0c91ae6c421fbd078eb198cd4729/users/98ba2c8a82264275b8635caae9c9737c/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:45.350302 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32342d-a1f5-4ee3-8b8c-ddb186ce618a None admin] RBAC: Authorizing `identity:create_grant(project_id=09c1bb7fd3204dffae9576cd2681a4a3, user_id=98ba2c8a82264275b8635caae9c9737c, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:45.351761 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32342d-a1f5-4ee3-8b8c-ddb186ce618a None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:45.352471 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50b77e-3545-4b24-bc12-f4114729851b None admin] RBAC: Authorizing `identity:create_grant(project_id=544b0c91ae6c421fbd078eb198cd4729, user_id=98ba2c8a82264275b8635caae9c9737c, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:45.354104 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50b77e-3545-4b24-bc12-f4114729851b None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:45.420363 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 63/134] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:23:45 2022] PUT /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3/users/98ba2c8a82264275b8635caae9c9737c/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:23:45.428460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32342d-a1f5-4ee3-8b8c-ddb186ce618a None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:45.485456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:45.494500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 72/135] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:23:45 2022] PUT /identity/v3/projects/544b0c91ae6c421fbd078eb198cd4729/users/98ba2c8a82264275b8635caae9c9737c/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:23:45.497073 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50b77e-3545-4b24-bc12-f4114729851b None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:45.606938 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:45.616228 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32342d-a1f5-4ee3-8b8c-ddb186ce618a None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:45.659408 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a6689-65d3-4339-a62b-35cd7afb6878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:45.660287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a6689-65d3-4339-a62b-35cd7afb6878 None admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:45.660676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a6689-65d3-4339-a62b-35cd7afb6878 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:45.661057 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a6689-65d3-4339-a62b-35cd7afb6878 None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3/users/b7cd207626714450952ac5fa63cc9cad/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:45.663784 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a6689-65d3-4339-a62b-35cd7afb6878 None admin] RBAC: Authorizing `identity:create_grant(project_id=09c1bb7fd3204dffae9576cd2681a4a3, user_id=b7cd207626714450952ac5fa63cc9cad, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:45.664746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a6689-65d3-4339-a62b-35cd7afb6878 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:45.666763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50b77e-3545-4b24-bc12-f4114729851b None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:45.710483 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9243b6-550c-453b-be59-077043090e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:45.711780 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9243b6-550c-453b-be59-077043090e5f None admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:45.711780 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9243b6-550c-453b-be59-077043090e5f None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:45.711780 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9243b6-550c-453b-be59-077043090e5f None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3/users/98ba2c8a82264275b8635caae9c9737c/roles/733cd39702ab4cbc860e34886dbb165f` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:45.714150 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9243b6-550c-453b-be59-077043090e5f None admin] RBAC: Authorizing `identity:create_grant(project_id=09c1bb7fd3204dffae9576cd2681a4a3, user_id=98ba2c8a82264275b8635caae9c9737c, role_id=733cd39702ab4cbc860e34886dbb165f)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:45.716230 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9243b6-550c-453b-be59-077043090e5f None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:45.747470 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 64/136] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:23:45 2022] PUT /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3/users/b7cd207626714450952ac5fa63cc9cad/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:23:45.748985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836e736-b904-45e8-bf0d-62a4f5f92b74 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:45.749278 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836e736-b904-45e8-bf0d-62a4f5f92b74 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:45.749556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836e736-b904-45e8-bf0d-62a4f5f92b74 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:45.750994 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 65/137] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:45.757305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87757164-7a27-49b3-bcbb-7e2bf2d29dfe None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:45.757626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87757164-7a27-49b3-bcbb-7e2bf2d29dfe None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:45.757871 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87757164-7a27-49b3-bcbb-7e2bf2d29dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:45.780349 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-87757164-7a27-49b3-bcbb-7e2bf2d29dfe None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:45.785990 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-87757164-7a27-49b3-bcbb-7e2bf2d29dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:45.816094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 73/138] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:23:45 2022] PUT /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3/users/98ba2c8a82264275b8635caae9c9737c/roles/733cd39702ab4cbc860e34886dbb165f => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:23:45.882454 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 66/139] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:23:46.029491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2bca1-d4d5-464c-8f7b-3e53562387ef None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:46.029679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2bca1-d4d5-464c-8f7b-3e53562387ef None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:46.029963 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2bca1-d4d5-464c-8f7b-3e53562387ef None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:46.031399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 74/140] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:23:46.036996 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6f1c6-2108-4a76-93e4-e200ccb86751 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:46.037285 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6f1c6-2108-4a76-93e4-e200ccb86751 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:46.037623 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6f1c6-2108-4a76-93e4-e200ccb86751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:46.064339 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-ecc6f1c6-2108-4a76-93e4-e200ccb86751 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:46.065559 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-ecc6f1c6-2108-4a76-93e4-e200ccb86751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:46.146203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 67/141] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:46.231454 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2bca1-d4d5-464c-8f7b-3e53562387ef None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:46.304558 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:46.357993 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2bca1-d4d5-464c-8f7b-3e53562387ef None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:46.399457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9bc14-a639-47cf-8ff1-2555090633a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:46.400175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9bc14-a639-47cf-8ff1-2555090633a5 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:46.400459 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9bc14-a639-47cf-8ff1-2555090633a5 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:46.400734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9bc14-a639-47cf-8ff1-2555090633a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:46.401865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9bc14-a639-47cf-8ff1-2555090633a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:46.404247 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9bc14-a639-47cf-8ff1-2555090633a5 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:46.405860 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 75/142] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:23:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:46.411573 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6f1c6-2108-4a76-93e4-e200ccb86751 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:46.435202 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:46.439482 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9bc14-a639-47cf-8ff1-2555090633a5 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:46.601665 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6f1c6-2108-4a76-93e4-e200ccb86751 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:46.619794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9bc14-a639-47cf-8ff1-2555090633a5 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:46.665895 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c2360-9d17-4635-b0a9-f068d5d124e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:46.666568 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c2360-9d17-4635-b0a9-f068d5d124e8 None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:46.666841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c2360-9d17-4635-b0a9-f068d5d124e8 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:46.667112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c2360-9d17-4635-b0a9-f068d5d124e8 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:46.667776 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c2360-9d17-4635-b0a9-f068d5d124e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:46.670898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c2360-9d17-4635-b0a9-f068d5d124e8 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:46.671897 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:46.705529 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca81d8-5c79-43af-b471-09ffad74294b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:46.706543 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca81d8-5c79-43af-b471-09ffad74294b None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:46.707027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca81d8-5c79-43af-b471-09ffad74294b None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:46.707587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca81d8-5c79-43af-b471-09ffad74294b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:46.736767 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ca81d8-5c79-43af-b471-09ffad74294b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:46.743535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-04ca81d8-5c79-43af-b471-09ffad74294b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:23:46.743535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:46.743535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:46.743535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:46.743535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:46.743535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:46.743535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:46.743535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:46.743535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:46.743535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:46.743535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:46.743535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:46.743535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:46.743535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:46.743535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:46.743535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:46.744697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:46.744697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:46.744697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:46.744697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:46.744697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:46.744697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:46.744697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:46.744697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:46.744697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:46.744697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:46.744697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:46.744697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:46.744697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:46.744697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:46.744697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:46.744697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:46.745508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:46.745508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:46.745508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:46.745508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:46.745508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:46.745508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:46.745508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:46.745508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:46.745508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:46.745508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:46.745508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:23:46.745508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:23:46.748376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 76/143] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:23:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:23:46.753887 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca81d8-5c79-43af-b471-09ffad74294b None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:46.803156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 68/144] 10.222.0.59 () {60 vars in 1132 bytes} [Mon Aug 29 22:23:46 2022] POST /identity/v3/users => generated 283 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:23:46.939261 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca81d8-5c79-43af-b471-09ffad74294b None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:46.997323 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b34d4-86b9-4713-bfef-0992ffa2d900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:46.998007 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b34d4-86b9-4713-bfef-0992ffa2d900 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:46.998284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b34d4-86b9-4713-bfef-0992ffa2d900 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:46.998551 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b34d4-86b9-4713-bfef-0992ffa2d900 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:46.999238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b34d4-86b9-4713-bfef-0992ffa2d900 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:46.999538 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b34d4-86b9-4713-bfef-0992ffa2d900 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:47.000995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b34d4-86b9-4713-bfef-0992ffa2d900 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:47.010504 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 77/145] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:23:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:47.016916 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c2360-9d17-4635-b0a9-f068d5d124e8 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:47.165527 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:47.212253 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c2360-9d17-4635-b0a9-f068d5d124e8 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:47.259936 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62110e1b-7383-4eea-af9d-65f43c1bb6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:47.260647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62110e1b-7383-4eea-af9d-65f43c1bb6ef None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:47.261041 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62110e1b-7383-4eea-af9d-65f43c1bb6ef None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:47.261445 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62110e1b-7383-4eea-af9d-65f43c1bb6ef None admin] PATH_INFO: `/v3/users/cf4fb11822bf402a8fb5b4947411e894` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:47.264967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62110e1b-7383-4eea-af9d-65f43c1bb6ef None admin] RBAC: Authorizing `identity:get_user(user_id=cf4fb11822bf402a8fb5b4947411e894)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:47.266681 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62110e1b-7383-4eea-af9d-65f43c1bb6ef None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:47.268578 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 69/146] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:47 2022] GET /identity/v3/users/cf4fb11822bf402a8fb5b4947411e894 => generated 289 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:47.280764 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b34d4-86b9-4713-bfef-0992ffa2d900 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:47.350471 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:47.421692 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b34d4-86b9-4713-bfef-0992ffa2d900 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:47.470466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e648a1-5ba6-4a86-a109-f8889b83ab04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:47.471139 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e648a1-5ba6-4a86-a109-f8889b83ab04 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:47.471486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e648a1-5ba6-4a86-a109-f8889b83ab04 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:47.471793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e648a1-5ba6-4a86-a109-f8889b83ab04 None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:47.473148 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e648a1-5ba6-4a86-a109-f8889b83ab04 None admin] RBAC: Authorizing `identity:get_project(project_id=09c1bb7fd3204dffae9576cd2681a4a3)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:47.474926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e648a1-5ba6-4a86-a109-f8889b83ab04 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:47.476961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 78/147] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:23:47 2022] GET /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:47.483856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62110e1b-7383-4eea-af9d-65f43c1bb6ef None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:47.657730 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62110e1b-7383-4eea-af9d-65f43c1bb6ef None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:47.703776 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045c3c5-6479-4ae5-9e2a-d19bb5dc5cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:47.704478 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045c3c5-6479-4ae5-9e2a-d19bb5dc5cfc None admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:47.704756 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045c3c5-6479-4ae5-9e2a-d19bb5dc5cfc None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:47.705027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045c3c5-6479-4ae5-9e2a-d19bb5dc5cfc None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3/users/cf4fb11822bf402a8fb5b4947411e894/roles/d487417153354d90853ba306bd1edb13` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:47.709444 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045c3c5-6479-4ae5-9e2a-d19bb5dc5cfc None admin] RBAC: Authorizing `identity:create_grant(project_id=09c1bb7fd3204dffae9576cd2681a4a3, user_id=cf4fb11822bf402a8fb5b4947411e894, role_id=d487417153354d90853ba306bd1edb13)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:47.710891 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045c3c5-6479-4ae5-9e2a-d19bb5dc5cfc None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:47.822393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:47.827929 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 70/148] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:23:47 2022] PUT /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3/users/cf4fb11822bf402a8fb5b4947411e894/roles/d487417153354d90853ba306bd1edb13 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:23:48.078494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fdb7ad-02b5-4cbd-a58e-cc007b190f74 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:48.078971 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fdb7ad-02b5-4cbd-a58e-cc007b190f74 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:48.078971 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fdb7ad-02b5-4cbd-a58e-cc007b190f74 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:48.080559 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 79/149] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:48.086739 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32626bd8-7052-4fb3-b3c5-25daae624887 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:48.087108 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32626bd8-7052-4fb3-b3c5-25daae624887 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:48.087386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32626bd8-7052-4fb3-b3c5-25daae624887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:48.109508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-32626bd8-7052-4fb3-b3c5-25daae624887 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:48.110710 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-32626bd8-7052-4fb3-b3c5-25daae624887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:48.118443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4a6b8-1c33-405c-b213-d882eeb54e9b None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:48.118717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4a6b8-1c33-405c-b213-d882eeb54e9b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:48.118985 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4a6b8-1c33-405c-b213-d882eeb54e9b None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:48.120347 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 80/150] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:23:48.126216 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2371705-8b12-4e8f-aab8-43365b975c66 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:48.126500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2371705-8b12-4e8f-aab8-43365b975c66 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:48.126773 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2371705-8b12-4e8f-aab8-43365b975c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:48.161170 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-d2371705-8b12-4e8f-aab8-43365b975c66 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:48.162413 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d2371705-8b12-4e8f-aab8-43365b975c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:48.218691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 71/151] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:48.255780 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 81/152] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:23:48.735020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32626bd8-7052-4fb3-b3c5-25daae624887 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:48.788852 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2371705-8b12-4e8f-aab8-43365b975c66 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:48.884535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:48.924574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32626bd8-7052-4fb3-b3c5-25daae624887 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:48.947814 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2371705-8b12-4e8f-aab8-43365b975c66 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:48.987042 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857c3e7-fe1f-46a5-8b54-c9a75e60462c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:48.988219 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857c3e7-fe1f-46a5-8b54-c9a75e60462c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:48.988500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857c3e7-fe1f-46a5-8b54-c9a75e60462c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:48.988908 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857c3e7-fe1f-46a5-8b54-c9a75e60462c None admin] PATH_INFO: `/v3/roles/member` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:49.001794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857c3e7-fe1f-46a5-8b54-c9a75e60462c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:49.006813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-4857c3e7-fe1f-46a5-8b54-c9a75e60462c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:23:49.006813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:49.006813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:49.006813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:49.006813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:49.006813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:49.006813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:49.006813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:49.006813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:49.006813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:49.006813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:49.006813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:49.006813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:49.006813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:49.006813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:49.006813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:49.007675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:49.007675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:49.007675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:49.007675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:49.007675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:49.007675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:49.007675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:49.007675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:49.007675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:49.007675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:49.007675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:49.007675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:49.007675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:49.007675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:49.007675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:49.007675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:49.008460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:49.008460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:49.008460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:49.008460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:49.008460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:49.008460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:49.008460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:49.008460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:49.008460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:49.008460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:49.008460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:23:49.008460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:23:49.008460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 82/153] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:23:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:23:49.014928 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857c3e7-fe1f-46a5-8b54-c9a75e60462c None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:49.040526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:49.061052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a26195-8dab-4a1e-8639-76b406742f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:49.061695 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a26195-8dab-4a1e-8639-76b406742f0a None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:49.062008 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a26195-8dab-4a1e-8639-76b406742f0a None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:49.062237 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a26195-8dab-4a1e-8639-76b406742f0a None admin] PATH_INFO: `/v3/roles/member` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:49.120834 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a26195-8dab-4a1e-8639-76b406742f0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:49.126460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-46a26195-8dab-4a1e-8639-76b406742f0a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:23:49.126460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:49.126460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:49.126460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:49.126460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:49.126460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:49.126460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:49.126460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:49.126460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:49.126460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:49.126460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:49.126460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:49.126460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:49.126460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:49.126460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:49.126460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:49.127421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:49.127421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:49.127421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:49.127421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:49.127421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:49.127421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:49.127421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:49.127421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:49.127421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:49.127421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:49.127421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:49.127421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:49.127421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:49.127421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:49.127421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:49.127421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:49.128263 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:49.128263 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:49.128263 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:49.128263 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:49.128263 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:49.128263 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:49.128263 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:49.128263 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:49.128263 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:49.128263 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:49.128263 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:23:49.128263 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:23:49.128263 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 72/154] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:23:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:23:49.129413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ce37b-a9d3-4d36-b89e-b3591d771220 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:49.129580 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ce37b-a9d3-4d36-b89e-b3591d771220 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:49.130472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ce37b-a9d3-4d36-b89e-b3591d771220 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:49.135617 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 73/155] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:49.136374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ce37b-a9d3-4d36-b89e-b3591d771220 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:49.212728 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857c3e7-fe1f-46a5-8b54-c9a75e60462c None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:49.224495 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:49.287488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f659c-d216-452a-aca9-51dc862e1cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:49.288033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f659c-d216-452a-aca9-51dc862e1cb9 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:49.288345 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f659c-d216-452a-aca9-51dc862e1cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:49.288625 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f659c-d216-452a-aca9-51dc862e1cb9 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:49.289548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f659c-d216-452a-aca9-51dc862e1cb9 None admin] RBAC: Adding query filter params (name=member) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:49.289861 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f659c-d216-452a-aca9-51dc862e1cb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:49.291281 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f659c-d216-452a-aca9-51dc862e1cb9 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:49.307796 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 83/156] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:23:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:23:49.309425 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83044d-a7f1-4eca-b255-9682851df449 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:49.309773 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83044d-a7f1-4eca-b255-9682851df449 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:49.310048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83044d-a7f1-4eca-b255-9682851df449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:49.347397 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-ae83044d-a7f1-4eca-b255-9682851df449 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:49.348649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ae83044d-a7f1-4eca-b255-9682851df449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:49.412937 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ce37b-a9d3-4d36-b89e-b3591d771220 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:49.500574 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 84/157] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:49.502357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83044d-a7f1-4eca-b255-9682851df449 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:49.506642 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d7001-35c9-442d-9c11-56296baa1e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:49.507543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d7001-35c9-442d-9c11-56296baa1e90 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:49.508071 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d7001-35c9-442d-9c11-56296baa1e90 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:49.508661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d7001-35c9-442d-9c11-56296baa1e90 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:49.509728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d7001-35c9-442d-9c11-56296baa1e90 None admin] RBAC: Adding query filter params (name=member) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:49.510507 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d7001-35c9-442d-9c11-56296baa1e90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:49.513806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d7001-35c9-442d-9c11-56296baa1e90 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:49.540514 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 74/158] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:23:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:49.542927 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b57d9-f2f2-44b9-a5d0-05f29fe585c8 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:49.548038 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b57d9-f2f2-44b9-a5d0-05f29fe585c8 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:49.549981 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b57d9-f2f2-44b9-a5d0-05f29fe585c8 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:49.552657 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 75/159] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:49 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:49.553815 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b57d9-f2f2-44b9-a5d0-05f29fe585c8 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:49.684590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:49.737692 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:49.742818 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83044d-a7f1-4eca-b255-9682851df449 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:49.760774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b57d9-f2f2-44b9-a5d0-05f29fe585c8 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:49.818340 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb35ca6-7a6e-4be5-9565-e241b8b67637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:49.819362 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb35ca6-7a6e-4be5-9565-e241b8b67637 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:49.819836 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb35ca6-7a6e-4be5-9565-e241b8b67637 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:49.820347 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb35ca6-7a6e-4be5-9565-e241b8b67637 None admin] PATH_INFO: `/v3/users/98ba2c8a82264275b8635caae9c9737c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:49.824967 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb35ca6-7a6e-4be5-9565-e241b8b67637 None admin] RBAC: Authorizing `identity:get_user(user_id=98ba2c8a82264275b8635caae9c9737c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:49.826862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb35ca6-7a6e-4be5-9565-e241b8b67637 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:49.828847 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 85/160] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:49 2022] GET /identity/v3/users/98ba2c8a82264275b8635caae9c9737c => generated 275 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:49.830208 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb303f-a8bd-4c1a-9f6d-df1ea3f6da6f None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:49.830706 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb303f-a8bd-4c1a-9f6d-df1ea3f6da6f None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:49.831181 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb303f-a8bd-4c1a-9f6d-df1ea3f6da6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:49.842006 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05850100-fc90-4ef5-921e-4cdcbb162a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:49.842683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05850100-fc90-4ef5-921e-4cdcbb162a2a None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:49.842961 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05850100-fc90-4ef5-921e-4cdcbb162a2a None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:49.843231 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05850100-fc90-4ef5-921e-4cdcbb162a2a None admin] PATH_INFO: `/v3/users/98ba2c8a82264275b8635caae9c9737c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:49.844420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05850100-fc90-4ef5-921e-4cdcbb162a2a None admin] RBAC: Authorizing `identity:get_user(user_id=98ba2c8a82264275b8635caae9c9737c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:49.845867 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05850100-fc90-4ef5-921e-4cdcbb162a2a None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:49.852632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 76/161] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:49 2022] GET /identity/v3/users/98ba2c8a82264275b8635caae9c9737c => generated 275 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:49.853578 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05850100-fc90-4ef5-921e-4cdcbb162a2a None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:49.862076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-6dcb303f-a8bd-4c1a-9f6d-df1ea3f6da6f None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:49.863616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6dcb303f-a8bd-4c1a-9f6d-df1ea3f6da6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:50.001793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 86/162] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:50.002523 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb303f-a8bd-4c1a-9f6d-df1ea3f6da6f None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:50.110266 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05850100-fc90-4ef5-921e-4cdcbb162a2a None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:50.145216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:50.175790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9c60f-913e-41b7-bdbf-1558e687b0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:50.176432 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9c60f-913e-41b7-bdbf-1558e687b0a7 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:50.176714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9c60f-913e-41b7-bdbf-1558e687b0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:50.176987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9c60f-913e-41b7-bdbf-1558e687b0a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:50.216602 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:50.227930 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb303f-a8bd-4c1a-9f6d-df1ea3f6da6f None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:50.309386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9c60f-913e-41b7-bdbf-1558e687b0a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:50.310894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9c60f-913e-41b7-bdbf-1558e687b0a7 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:50.326536 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd17ea2-d040-45ba-8593-6b942e9e93cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:50.327166 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd17ea2-d040-45ba-8593-6b942e9e93cb None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:50.327443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd17ea2-d040-45ba-8593-6b942e9e93cb None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:50.327714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd17ea2-d040-45ba-8593-6b942e9e93cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:50.484271 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd17ea2-d040-45ba-8593-6b942e9e93cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:50.485757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd17ea2-d040-45ba-8593-6b942e9e93cb None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:50.512284 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 77/163] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:23:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 659 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:23:50.513379 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9c60f-913e-41b7-bdbf-1558e687b0a7 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:50.556835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:50.648869 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 87/164] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:23:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:23:50.650520 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd17ea2-d040-45ba-8593-6b942e9e93cb None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:50.660680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:50.763027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd17ea2-d040-45ba-8593-6b942e9e93cb None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:50.786436 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9c60f-913e-41b7-bdbf-1558e687b0a7 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:50.843095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c10b75-e231-41ea-882b-74da582eb852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:50.843744 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c10b75-e231-41ea-882b-74da582eb852 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:50.844020 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c10b75-e231-41ea-882b-74da582eb852 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:50.844321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c10b75-e231-41ea-882b-74da582eb852 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:50.871778 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c10b75-e231-41ea-882b-74da582eb852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:50.877011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-72c10b75-e231-41ea-882b-74da582eb852 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:23:50.877011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:50.877011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:50.877011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:50.877011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:50.877011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:50.877011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:50.877011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:50.877011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:50.877011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:50.877011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:50.877011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:50.877011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:50.877011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:50.877011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:50.877011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:50.877858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:50.877858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:50.877858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:50.877858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:50.877858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:50.877858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:50.877858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:50.877858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:50.877858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:50.877858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:50.877858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:50.877858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:50.877858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:50.877858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:50.877858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:50.877858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:50.878611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:50.878611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:50.878611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:50.878611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:50.878611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:50.878611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:50.878611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:50.878611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:50.878611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:50.878611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:50.878611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:23:50.878611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:23:50.878611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 88/165] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:23:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:23:50.879459 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c10b75-e231-41ea-882b-74da582eb852 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:50.880907 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be435f91-371a-4549-852e-866dfe83769b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:50.881562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be435f91-371a-4549-852e-866dfe83769b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:50.881864 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be435f91-371a-4549-852e-866dfe83769b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:50.882136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be435f91-371a-4549-852e-866dfe83769b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:50.927737 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be435f91-371a-4549-852e-866dfe83769b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:50.932759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-be435f91-371a-4549-852e-866dfe83769b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:23:50.932759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:50.932759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:50.932759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:50.932759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:50.932759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:50.932759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:50.932759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:50.932759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:50.932759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:50.932759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:50.932759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:50.932759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:50.932759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:50.932759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:50.932759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:50.933691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:50.933691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:50.933691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:50.933691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:50.933691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:50.933691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:50.933691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:50.933691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:50.933691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:50.933691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:50.933691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:50.933691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:50.933691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:50.933691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:50.933691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:50.933691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:50.934601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:50.934601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:50.934601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:50.934601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:50.934601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:50.934601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:50.934601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:50.934601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:50.934601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:50.934601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:50.934601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:23:50.934601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:23:50.934601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 78/166] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:23:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:23:50.935249 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be435f91-371a-4549-852e-866dfe83769b None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:50.972501 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:51.018349 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c10b75-e231-41ea-882b-74da582eb852 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:51.058488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce318672-4857-45c1-beac-2c3b49d609c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:51.059658 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce318672-4857-45c1-beac-2c3b49d609c3 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:51.060260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce318672-4857-45c1-beac-2c3b49d609c3 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:51.060722 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce318672-4857-45c1-beac-2c3b49d609c3 None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:51.062132 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce318672-4857-45c1-beac-2c3b49d609c3 None admin] RBAC: Authorizing `identity:get_project(project_id=09c1bb7fd3204dffae9576cd2681a4a3)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:51.064426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce318672-4857-45c1-beac-2c3b49d609c3 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:51.066759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 89/167] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:23:50 2022] GET /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:51.067730 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce318672-4857-45c1-beac-2c3b49d609c3 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:51.187761 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce318672-4857-45c1-beac-2c3b49d609c3 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:51.215566 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:51.231458 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:51.233892 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145af08-4dc9-4c33-ad5e-0b8304597f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:51.235323 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145af08-4dc9-4c33-ad5e-0b8304597f6c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:51.235798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145af08-4dc9-4c33-ad5e-0b8304597f6c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:51.236321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145af08-4dc9-4c33-ad5e-0b8304597f6c None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:51.237324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145af08-4dc9-4c33-ad5e-0b8304597f6c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:51.237820 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145af08-4dc9-4c33-ad5e-0b8304597f6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:51.239542 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145af08-4dc9-4c33-ad5e-0b8304597f6c None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:51.250767 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 90/168] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:23:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:51.251916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145af08-4dc9-4c33-ad5e-0b8304597f6c None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:51.274315 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be435f91-371a-4549-852e-866dfe83769b None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:51.357858 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71defe9-9104-4127-86dd-7d035f76c3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:51.358503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71defe9-9104-4127-86dd-7d035f76c3a6 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:51.358944 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71defe9-9104-4127-86dd-7d035f76c3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:51.359419 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71defe9-9104-4127-86dd-7d035f76c3a6 None admin] PATH_INFO: `/v3/projects/544b0c91ae6c421fbd078eb198cd4729` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:51.360884 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71defe9-9104-4127-86dd-7d035f76c3a6 None admin] RBAC: Authorizing `identity:get_project(project_id=544b0c91ae6c421fbd078eb198cd4729)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:51.362665 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71defe9-9104-4127-86dd-7d035f76c3a6 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:51.365072 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 79/169] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:23:50 2022] GET /identity/v3/projects/544b0c91ae6c421fbd078eb198cd4729 => generated 312 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:51.366599 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c180da-d6d8-4568-8674-86548e7e9244 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:51.367046 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145af08-4dc9-4c33-ad5e-0b8304597f6c None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:51.367170 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c180da-d6d8-4568-8674-86548e7e9244 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:51.367560 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c180da-d6d8-4568-8674-86548e7e9244 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:51.369020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 80/170] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:51.370057 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c180da-d6d8-4568-8674-86548e7e9244 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:51.410851 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eac68d-5cc1-4651-82cb-b4be2b5bf55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:51.411456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eac68d-5cc1-4651-82cb-b4be2b5bf55c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:51.411728 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eac68d-5cc1-4651-82cb-b4be2b5bf55c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:51.411993 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eac68d-5cc1-4651-82cb-b4be2b5bf55c None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:51.412864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eac68d-5cc1-4651-82cb-b4be2b5bf55c None admin] RBAC: Adding query filter params (name=admin) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:51.413158 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eac68d-5cc1-4651-82cb-b4be2b5bf55c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:51.415338 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eac68d-5cc1-4651-82cb-b4be2b5bf55c None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:51.425906 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 91/171] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:23:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:51.431257 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eac68d-5cc1-4651-82cb-b4be2b5bf55c None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:51.487793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:51.494031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c180da-d6d8-4568-8674-86548e7e9244 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:51.532829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d82b4-dc7a-4f44-b34c-1aebd7d0c8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:51.533943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d82b4-dc7a-4f44-b34c-1aebd7d0c8d8 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:51.534471 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d82b4-dc7a-4f44-b34c-1aebd7d0c8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:51.535000 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d82b4-dc7a-4f44-b34c-1aebd7d0c8d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:51.536032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d82b4-dc7a-4f44-b34c-1aebd7d0c8d8 None admin] RBAC: Adding query filter params (role.id=454bd427dfc04030a4ec2b0e8a1ed237, scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3, user.id=98ba2c8a82264275b8635caae9c9737c) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:51.536676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d82b4-dc7a-4f44-b34c-1aebd7d0c8d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:51.538379 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d82b4-dc7a-4f44-b34c-1aebd7d0c8d8 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:51.546530 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 81/172] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:23:51 2022] GET /identity/v3/role_assignments?user.id=98ba2c8a82264275b8635caae9c9737c&scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3&role.id=454bd427dfc04030a4ec2b0e8a1ed237 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:51.548084 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e983085-d1db-4a36-8c04-3a82c0aee810 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:51.548596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e983085-d1db-4a36-8c04-3a82c0aee810 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:51.549062 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e983085-d1db-4a36-8c04-3a82c0aee810 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:51.550509 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 82/173] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:51.552013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad21fb-adc4-4fd4-b161-258c69224f6a None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:51.552643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad21fb-adc4-4fd4-b161-258c69224f6a None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:51.553111 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad21fb-adc4-4fd4-b161-258c69224f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:51.555784 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eac68d-5cc1-4651-82cb-b4be2b5bf55c None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:51.584991 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-bcad21fb-adc4-4fd4-b161-258c69224f6a None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:51.586572 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-bcad21fb-adc4-4fd4-b161-258c69224f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:51.600556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:51.618003 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d5348-4163-4e7e-b221-e02a1cd826a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:51.619539 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d5348-4163-4e7e-b221-e02a1cd826a1 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:51.620050 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d5348-4163-4e7e-b221-e02a1cd826a1 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:51.620534 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d5348-4163-4e7e-b221-e02a1cd826a1 None admin] PATH_INFO: `/v3/users/98ba2c8a82264275b8635caae9c9737c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:51.624739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416d5348-4163-4e7e-b221-e02a1cd826a1 None admin] RBAC: Authorizing `identity:get_user(user_id=98ba2c8a82264275b8635caae9c9737c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:51.626159 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416d5348-4163-4e7e-b221-e02a1cd826a1 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:51.628907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 92/174] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:51 2022] GET /identity/v3/users/98ba2c8a82264275b8635caae9c9737c => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:51.630352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d5348-4163-4e7e-b221-e02a1cd826a1 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:51.675424 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 83/175] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:51.685423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad21fb-adc4-4fd4-b161-258c69224f6a None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:51.770240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d5348-4163-4e7e-b221-e02a1cd826a1 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:51.794107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:51.816025 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2acb8-be59-425b-ba22-e2172cd1e5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:51.816812 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2acb8-be59-425b-ba22-e2172cd1e5b5 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:51.817201 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2acb8-be59-425b-ba22-e2172cd1e5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:51.818592 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2acb8-be59-425b-ba22-e2172cd1e5b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:51.819441 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2acb8-be59-425b-ba22-e2172cd1e5b5 None admin] RBAC: Adding query filter params (role.id=454bd427dfc04030a4ec2b0e8a1ed237, scope.project.id=544b0c91ae6c421fbd078eb198cd4729, user.id=98ba2c8a82264275b8635caae9c9737c) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:51.819853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2acb8-be59-425b-ba22-e2172cd1e5b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:51.822043 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2acb8-be59-425b-ba22-e2172cd1e5b5 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:51.831574 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 93/176] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:23:51 2022] GET /identity/v3/role_assignments?user.id=98ba2c8a82264275b8635caae9c9737c&scope.project.id=544b0c91ae6c421fbd078eb198cd4729&role.id=454bd427dfc04030a4ec2b0e8a1ed237 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:51.833053 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facd1d7-ac69-4b4f-8bd5-72ccaeca7cbc None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:51.836348 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facd1d7-ac69-4b4f-8bd5-72ccaeca7cbc None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:51.836348 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facd1d7-ac69-4b4f-8bd5-72ccaeca7cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:51.838889 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad21fb-adc4-4fd4-b161-258c69224f6a None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:51.859682 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-2facd1d7-ac69-4b4f-8bd5-72ccaeca7cbc None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:51.861069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-2facd1d7-ac69-4b4f-8bd5-72ccaeca7cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:51.883969 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b58d4-4e64-40b2-8d3d-ee9f2267f727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:51.884915 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b58d4-4e64-40b2-8d3d-ee9f2267f727 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:51.885362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b58d4-4e64-40b2-8d3d-ee9f2267f727 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:51.885915 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b58d4-4e64-40b2-8d3d-ee9f2267f727 None admin] PATH_INFO: `/v3/users/b7cd207626714450952ac5fa63cc9cad` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:51.888034 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b58d4-4e64-40b2-8d3d-ee9f2267f727 None admin] RBAC: Authorizing `identity:get_user(user_id=b7cd207626714450952ac5fa63cc9cad)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:51.889716 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b58d4-4e64-40b2-8d3d-ee9f2267f727 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:51.892042 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 84/177] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:51 2022] GET /identity/v3/users/b7cd207626714450952ac5fa63cc9cad => generated 247 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:51.897956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1646f9a-a4d7-46b5-acab-e6e4ea266390 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:51.898428 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1646f9a-a4d7-46b5-acab-e6e4ea266390 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:51.898847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1646f9a-a4d7-46b5-acab-e6e4ea266390 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:51.900462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 85/178] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:51.901448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1646f9a-a4d7-46b5-acab-e6e4ea266390 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:51.919750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:51.961404 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 94/179] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:51.962912 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef584c-62df-456c-936b-4a893df00101 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:51.963313 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef584c-62df-456c-936b-4a893df00101 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:51.963678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef584c-62df-456c-936b-4a893df00101 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:51.965861 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 95/180] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:51.966772 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef584c-62df-456c-936b-4a893df00101 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:52.027326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1646f9a-a4d7-46b5-acab-e6e4ea266390 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:52.071032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64114e6-5328-4eff-9d96-13fcd878324e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:52.071946 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64114e6-5328-4eff-9d96-13fcd878324e None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:52.072434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64114e6-5328-4eff-9d96-13fcd878324e None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:52.072900 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64114e6-5328-4eff-9d96-13fcd878324e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:52.091294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef584c-62df-456c-936b-4a893df00101 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:52.104001 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:52.131316 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373b062-798b-452e-911a-ec7a6d5ea17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:52.131929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373b062-798b-452e-911a-ec7a6d5ea17a None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:52.132288 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373b062-798b-452e-911a-ec7a6d5ea17a None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:52.132694 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373b062-798b-452e-911a-ec7a6d5ea17a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:52.174999 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64114e6-5328-4eff-9d96-13fcd878324e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:52.176752 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64114e6-5328-4eff-9d96-13fcd878324e None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:52.189309 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:52.217749 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373b062-798b-452e-911a-ec7a6d5ea17a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:52.219218 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373b062-798b-452e-911a-ec7a6d5ea17a None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:52.290098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 86/181] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:23:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:23:52.291649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d2f4c-63de-432f-a16e-37a3f0af1fb2 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:52.291925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d2f4c-63de-432f-a16e-37a3f0af1fb2 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:52.292215 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d2f4c-63de-432f-a16e-37a3f0af1fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:52.310461 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-2e2d2f4c-63de-432f-a16e-37a3f0af1fb2 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:52.311634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-2e2d2f4c-63de-432f-a16e-37a3f0af1fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:52.330930 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 96/182] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:23:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:23:52.332103 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a576b-c3f5-4ad2-a545-031d4db1f073 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:52.332426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a576b-c3f5-4ad2-a545-031d4db1f073 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:52.332731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a576b-c3f5-4ad2-a545-031d4db1f073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:52.350755 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-959a576b-c3f5-4ad2-a545-031d4db1f073 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:52.351946 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-959a576b-c3f5-4ad2-a545-031d4db1f073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:52.354666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:52.389241 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 87/183] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:52.390085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d2f4c-63de-432f-a16e-37a3f0af1fb2 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:52.424348 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 97/184] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:52.425351 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a576b-c3f5-4ad2-a545-031d4db1f073 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:52.439515 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:52.503866 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d2f4c-63de-432f-a16e-37a3f0af1fb2 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:52.532521 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a576b-c3f5-4ad2-a545-031d4db1f073 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:52.544161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45577b-26c3-4c6d-9283-bee6be273d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:52.545096 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45577b-26c3-4c6d-9283-bee6be273d75 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:52.545570 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45577b-26c3-4c6d-9283-bee6be273d75 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:52.546034 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45577b-26c3-4c6d-9283-bee6be273d75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:52.557796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb45577b-26c3-4c6d-9283-bee6be273d75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:52.563496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-fb45577b-26c3-4c6d-9283-bee6be273d75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:23:52.563496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:52.563496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:52.563496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:52.563496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:52.563496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:52.563496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:52.563496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:52.563496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:52.563496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:52.563496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:52.563496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:52.563496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:52.563496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:52.563496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:52.563496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:52.564400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:52.564400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:52.564400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:52.564400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:52.564400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:52.564400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:52.564400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:52.564400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:52.564400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:52.564400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:52.564400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:52.564400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:52.564400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:52.564400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:52.564400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:52.564400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:52.565193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:52.565193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:52.565193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:52.565193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:52.565193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:52.565193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:52.565193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:52.565193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:52.565193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:52.565193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:52.565193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:23:52.565193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:23:52.567151 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 88/185] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:23:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:23:52.568016 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45577b-26c3-4c6d-9283-bee6be273d75 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:52.570127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2353057-1ebe-4ff5-b704-66ba26ed86cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:52.571063 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2353057-1ebe-4ff5-b704-66ba26ed86cc None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:52.571559 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2353057-1ebe-4ff5-b704-66ba26ed86cc None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:52.572016 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2353057-1ebe-4ff5-b704-66ba26ed86cc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:52.581513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:52.584824 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2353057-1ebe-4ff5-b704-66ba26ed86cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:52.590976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-f2353057-1ebe-4ff5-b704-66ba26ed86cc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:23:52.590976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:52.590976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:52.590976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:52.590976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:52.590976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:52.590976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:52.590976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:52.590976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:52.590976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:52.590976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:52.590976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:52.590976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:52.590976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:52.590976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:52.590976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:52.596801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:52.596801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:52.596801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:52.596801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:52.596801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:52.596801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:52.596801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:52.596801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:52.596801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:52.596801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:52.596801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:52.596801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:52.596801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:52.596801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:52.596801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:52.596801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:52.597580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:52.597580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:52.597580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:52.597580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:52.597580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:52.597580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:52.597580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:52.597580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:52.597580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:52.597580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:52.597580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:23:52.597580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:23:52.597580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 98/186] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:23:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:23:52.597580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2353057-1ebe-4ff5-b704-66ba26ed86cc None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:52.679347 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:52.682528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45577b-26c3-4c6d-9283-bee6be273d75 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:52.707790 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2353057-1ebe-4ff5-b704-66ba26ed86cc None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:52.720967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92133e0f-5af3-4986-85fc-3655ee71d8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:52.721597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92133e0f-5af3-4986-85fc-3655ee71d8b2 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:52.721872 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92133e0f-5af3-4986-85fc-3655ee71d8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:52.722149 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92133e0f-5af3-4986-85fc-3655ee71d8b2 None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:52.723269 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92133e0f-5af3-4986-85fc-3655ee71d8b2 None admin] RBAC: Authorizing `identity:get_project(project_id=09c1bb7fd3204dffae9576cd2681a4a3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:52.724746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92133e0f-5af3-4986-85fc-3655ee71d8b2 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:52.726361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 89/187] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:23:52 2022] GET /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:52.727121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92133e0f-5af3-4986-85fc-3655ee71d8b2 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:52.743771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96dc8e9-b935-4f30-a013-01ede38f1172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:52.744462 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96dc8e9-b935-4f30-a013-01ede38f1172 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:52.744741 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96dc8e9-b935-4f30-a013-01ede38f1172 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:52.745014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96dc8e9-b935-4f30-a013-01ede38f1172 None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:52.746515 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96dc8e9-b935-4f30-a013-01ede38f1172 None admin] RBAC: Authorizing `identity:get_project(project_id=09c1bb7fd3204dffae9576cd2681a4a3)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:52.748094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96dc8e9-b935-4f30-a013-01ede38f1172 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:52.749927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 99/188] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:23:52 2022] GET /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:52.750619 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96dc8e9-b935-4f30-a013-01ede38f1172 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:52.817591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:52.833337 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92133e0f-5af3-4986-85fc-3655ee71d8b2 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:52.850662 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96dc8e9-b935-4f30-a013-01ede38f1172 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:52.868584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5bfb0d-9db1-4558-928d-9ba2919be3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:52.869194 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bfb0d-9db1-4558-928d-9ba2919be3f9 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:52.869468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bfb0d-9db1-4558-928d-9ba2919be3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:52.869735 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bfb0d-9db1-4558-928d-9ba2919be3f9 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:52.870393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5bfb0d-9db1-4558-928d-9ba2919be3f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:52.870983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5bfb0d-9db1-4558-928d-9ba2919be3f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:52.872425 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5bfb0d-9db1-4558-928d-9ba2919be3f9 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:52.880517 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 90/189] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:23:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:52.881345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5bfb0d-9db1-4558-928d-9ba2919be3f9 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:52.885103 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397946c-733c-4624-a883-a647ef56f861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:52.885908 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397946c-733c-4624-a883-a647ef56f861 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:52.886182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397946c-733c-4624-a883-a647ef56f861 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:52.886520 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397946c-733c-4624-a883-a647ef56f861 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:52.887176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397946c-733c-4624-a883-a647ef56f861 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:52.887512 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397946c-733c-4624-a883-a647ef56f861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:52.888896 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397946c-733c-4624-a883-a647ef56f861 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:52.901651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 100/190] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:23:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:52.901651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397946c-733c-4624-a883-a647ef56f861 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:52.917130 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:52.987615 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5bfb0d-9db1-4558-928d-9ba2919be3f9 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:53.003165 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397946c-733c-4624-a883-a647ef56f861 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:53.022160 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da0c96-201f-4ff1-a5ed-b73095ea4a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:53.022789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da0c96-201f-4ff1-a5ed-b73095ea4a82 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:53.023085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da0c96-201f-4ff1-a5ed-b73095ea4a82 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:53.023345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da0c96-201f-4ff1-a5ed-b73095ea4a82 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:53.031255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:53.033309 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da0c96-201f-4ff1-a5ed-b73095ea4a82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:53.036883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec997e-44b5-47e8-9d4e-9dcdbad761e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:53.037497 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec997e-44b5-47e8-9d4e-9dcdbad761e2 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:53.044556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec997e-44b5-47e8-9d4e-9dcdbad761e2 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:53.044556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec997e-44b5-47e8-9d4e-9dcdbad761e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:53.044753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-67da0c96-201f-4ff1-a5ed-b73095ea4a82 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:23:53.044753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:53.044753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:53.044753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:53.044753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:53.044753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:53.044753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:53.044753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:53.044753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:53.044753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:53.044753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:53.044753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:53.044753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:53.044753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:53.044753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:53.044753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:53.045619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:53.045619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:53.045619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:53.045619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:53.045619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:53.045619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:53.045619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:53.045619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:53.045619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:53.045619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:53.045619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:53.045619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:53.045619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:53.045619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:53.045619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:53.045619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:53.046380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:53.046380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:53.046380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:53.046380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:53.046380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:53.046380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:53.046380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:53.046380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:53.046380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:53.046380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:53.046380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:23:53.046380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:23:53.048353 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec997e-44b5-47e8-9d4e-9dcdbad761e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:53.048503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 91/191] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:23:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:23:53.048619 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec997e-44b5-47e8-9d4e-9dcdbad761e2 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:53.048744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da0c96-201f-4ff1-a5ed-b73095ea4a82 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:53.048905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 101/192] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:23:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:53.048905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec997e-44b5-47e8-9d4e-9dcdbad761e2 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:53.138515 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:53.147296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da0c96-201f-4ff1-a5ed-b73095ea4a82 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:53.151816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec997e-44b5-47e8-9d4e-9dcdbad761e2 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:53.181476 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88078a7-c68e-45e0-b0dc-2d9ba0355047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:53.182071 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88078a7-c68e-45e0-b0dc-2d9ba0355047 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:53.182349 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88078a7-c68e-45e0-b0dc-2d9ba0355047 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:53.182620 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88078a7-c68e-45e0-b0dc-2d9ba0355047 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:53.183500 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88078a7-c68e-45e0-b0dc-2d9ba0355047 None admin] RBAC: Adding query filter params (role.id=733cd39702ab4cbc860e34886dbb165f, scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3, user.id=98ba2c8a82264275b8635caae9c9737c) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:53.184001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88078a7-c68e-45e0-b0dc-2d9ba0355047 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:53.185496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88078a7-c68e-45e0-b0dc-2d9ba0355047 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:53.186400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952da827-eafa-4d85-8a82-2ebe09ac6c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:53.187001 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952da827-eafa-4d85-8a82-2ebe09ac6c1f None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:53.187282 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952da827-eafa-4d85-8a82-2ebe09ac6c1f None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:53.187557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952da827-eafa-4d85-8a82-2ebe09ac6c1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:53.188319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952da827-eafa-4d85-8a82-2ebe09ac6c1f None admin] RBAC: Adding query filter params (role.id=a836108447dc439e994611bc74d53d9c, scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3, user.id=b7cd207626714450952ac5fa63cc9cad) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:53.188622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952da827-eafa-4d85-8a82-2ebe09ac6c1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:53.190031 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952da827-eafa-4d85-8a82-2ebe09ac6c1f None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:53.191286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 92/193] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:23:53 2022] GET /identity/v3/role_assignments?user.id=98ba2c8a82264275b8635caae9c9737c&scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3&role.id=733cd39702ab4cbc860e34886dbb165f => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:53.192071 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88078a7-c68e-45e0-b0dc-2d9ba0355047 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:53.196042 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 102/194] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:23:53 2022] GET /identity/v3/role_assignments?user.id=b7cd207626714450952ac5fa63cc9cad&scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3&role.id=a836108447dc439e994611bc74d53d9c => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:53.197019 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952da827-eafa-4d85-8a82-2ebe09ac6c1f None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:53.293532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:53.341333 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952da827-eafa-4d85-8a82-2ebe09ac6c1f None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:53.342827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88078a7-c68e-45e0-b0dc-2d9ba0355047 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:53.381285 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037aa46-0645-4002-883b-f45ed888e1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:53.381485 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211944fc-594c-46d2-b131-09025b7bfb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:53.382050 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037aa46-0645-4002-883b-f45ed888e1ae None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:53.382413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037aa46-0645-4002-883b-f45ed888e1ae None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:53.382722 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211944fc-594c-46d2-b131-09025b7bfb63 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:53.382820 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037aa46-0645-4002-883b-f45ed888e1ae None admin] PATH_INFO: `/v3/users/970d0be8205f4a45a498452f92f1768c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:53.383337 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211944fc-594c-46d2-b131-09025b7bfb63 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:53.383882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211944fc-594c-46d2-b131-09025b7bfb63 None admin] PATH_INFO: `/v3/users/cf4fb11822bf402a8fb5b4947411e894` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:53.386027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211944fc-594c-46d2-b131-09025b7bfb63 None admin] RBAC: Authorizing `identity:get_user(user_id=cf4fb11822bf402a8fb5b4947411e894)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:53.388119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211944fc-594c-46d2-b131-09025b7bfb63 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:53.389834 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 103/195] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:53 2022] GET /identity/v3/users/cf4fb11822bf402a8fb5b4947411e894 => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:53.390543 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211944fc-594c-46d2-b131-09025b7bfb63 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:53.404130 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037aa46-0645-4002-883b-f45ed888e1ae None admin] RBAC: Authorizing `identity:get_user(user_id=970d0be8205f4a45a498452f92f1768c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:53.405700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037aa46-0645-4002-883b-f45ed888e1ae None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:53.407627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 93/196] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:53 2022] GET /identity/v3/users/970d0be8205f4a45a498452f92f1768c => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:53.408225 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037aa46-0645-4002-883b-f45ed888e1ae None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:53.431059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:53.510879 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211944fc-594c-46d2-b131-09025b7bfb63 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:53.525962 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037aa46-0645-4002-883b-f45ed888e1ae None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:53.540029 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:53.546180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea21bd8-3ad5-4262-8504-8bf6da0c4de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:53.546788 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea21bd8-3ad5-4262-8504-8bf6da0c4de4 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:53.547060 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea21bd8-3ad5-4262-8504-8bf6da0c4de4 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:53.547324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea21bd8-3ad5-4262-8504-8bf6da0c4de4 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:53.547975 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea21bd8-3ad5-4262-8504-8bf6da0c4de4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:53.548301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea21bd8-3ad5-4262-8504-8bf6da0c4de4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:53.550119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea21bd8-3ad5-4262-8504-8bf6da0c4de4 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:53.560032 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 104/197] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:23:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:53.560958 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea21bd8-3ad5-4262-8504-8bf6da0c4de4 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:53.565896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f0044-c393-471d-a36c-f1228ce6b5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:53.566501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f0044-c393-471d-a36c-f1228ce6b5a7 None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:53.566774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f0044-c393-471d-a36c-f1228ce6b5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:53.567148 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f0044-c393-471d-a36c-f1228ce6b5a7 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:53.567796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f0044-c393-471d-a36c-f1228ce6b5a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:53.569260 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f0044-c393-471d-a36c-f1228ce6b5a7 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:53.661885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea21bd8-3ad5-4262-8504-8bf6da0c4de4 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:53.677190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 94/198] 10.222.0.59 () {60 vars in 1132 bytes} [Mon Aug 29 22:23:53 2022] POST /identity/v3/users => generated 297 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:23:53.678119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f0044-c393-471d-a36c-f1228ce6b5a7 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:53.700933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9748a-3210-47e5-9efb-2a171d4a6dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:53.701639 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9748a-3210-47e5-9efb-2a171d4a6dc9 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:53.708114 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9748a-3210-47e5-9efb-2a171d4a6dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:53.708114 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9748a-3210-47e5-9efb-2a171d4a6dc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:53.762810 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:53.773306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:53.785151 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f9748a-3210-47e5-9efb-2a171d4a6dc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:53.786990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f9748a-3210-47e5-9efb-2a171d4a6dc9 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:53.803445 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f0044-c393-471d-a36c-f1228ce6b5a7 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:53.844902 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96439dab-0036-4dff-af5f-f3d77f2bcd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:53.845670 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96439dab-0036-4dff-af5f-f3d77f2bcd1c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:53.846068 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96439dab-0036-4dff-af5f-f3d77f2bcd1c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:53.846444 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96439dab-0036-4dff-af5f-f3d77f2bcd1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:53.909165 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 105/199] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:23:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:23:53.910101 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9748a-3210-47e5-9efb-2a171d4a6dc9 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:53.942960 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96439dab-0036-4dff-af5f-f3d77f2bcd1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:53.945317 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96439dab-0036-4dff-af5f-f3d77f2bcd1c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:54.031299 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:54.035794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:54.047626 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9748a-3210-47e5-9efb-2a171d4a6dc9 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:54.074029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 95/200] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:23:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:23:54.074830 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96439dab-0036-4dff-af5f-f3d77f2bcd1c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:54.096079 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f551d3-d189-4a9b-a652-80fb8fb2fced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:54.096794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f551d3-d189-4a9b-a652-80fb8fb2fced None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:54.097073 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f551d3-d189-4a9b-a652-80fb8fb2fced None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:54.097840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f551d3-d189-4a9b-a652-80fb8fb2fced None admin] PATH_INFO: `/v3/users/970d0be8205f4a45a498452f92f1768c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:54.099356 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f551d3-d189-4a9b-a652-80fb8fb2fced None admin] RBAC: Authorizing `identity:get_user(user_id=970d0be8205f4a45a498452f92f1768c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:54.101030 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f551d3-d189-4a9b-a652-80fb8fb2fced None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:54.103434 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 106/201] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:53 2022] GET /identity/v3/users/970d0be8205f4a45a498452f92f1768c => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:54.104433 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f551d3-d189-4a9b-a652-80fb8fb2fced None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:54.189497 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96439dab-0036-4dff-af5f-f3d77f2bcd1c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:54.223432 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f551d3-d189-4a9b-a652-80fb8fb2fced None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:54.229213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a23fb4-6e0f-44c4-a824-c989a3058d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:54.229844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a23fb4-6e0f-44c4-a824-c989a3058d16 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:54.230169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a23fb4-6e0f-44c4-a824-c989a3058d16 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:54.230444 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a23fb4-6e0f-44c4-a824-c989a3058d16 None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:54.231852 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a23fb4-6e0f-44c4-a824-c989a3058d16 None admin] RBAC: Authorizing `identity:get_project(project_id=09c1bb7fd3204dffae9576cd2681a4a3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:54.233525 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a23fb4-6e0f-44c4-a824-c989a3058d16 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:54.235589 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 96/202] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:23:54 2022] GET /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:54.236356 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a23fb4-6e0f-44c4-a824-c989a3058d16 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:54.262595 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c1c72-9608-4ee6-920f-662e1c78dc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:54.263254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c1c72-9608-4ee6-920f-662e1c78dc38 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:54.263534 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c1c72-9608-4ee6-920f-662e1c78dc38 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:54.263803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c1c72-9608-4ee6-920f-662e1c78dc38 None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:54.265019 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c1c72-9608-4ee6-920f-662e1c78dc38 None admin] RBAC: Authorizing `identity:get_project(project_id=7d4cfda3691c4a4ab82dda7546659db6)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:54.266452 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c1c72-9608-4ee6-920f-662e1c78dc38 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:54.268093 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 107/203] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:23:54 2022] GET /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:54.268898 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c1c72-9608-4ee6-920f-662e1c78dc38 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:54.277229 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:54.299408 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:54.352107 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a23fb4-6e0f-44c4-a824-c989a3058d16 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:54.380422 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c1c72-9608-4ee6-920f-662e1c78dc38 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:54.395201 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3141a2-476d-4f2e-813e-d1638d670f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:54.396344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3141a2-476d-4f2e-813e-d1638d670f92 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:54.396768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3141a2-476d-4f2e-813e-d1638d670f92 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:54.397204 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3141a2-476d-4f2e-813e-d1638d670f92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:54.418421 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c4abc-d615-4c3b-b43b-342e479508ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:54.419037 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c4abc-d615-4c3b-b43b-342e479508ae None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:54.419318 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c4abc-d615-4c3b-b43b-342e479508ae None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:54.419590 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c4abc-d615-4c3b-b43b-342e479508ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:54.420344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c4abc-d615-4c3b-b43b-342e479508ae None admin] RBAC: Adding query filter params (role.id=d487417153354d90853ba306bd1edb13, scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3, user.id=cf4fb11822bf402a8fb5b4947411e894) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:54.420758 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c4abc-d615-4c3b-b43b-342e479508ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:54.422230 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c4abc-d615-4c3b-b43b-342e479508ae None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:54.428520 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 108/204] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:23:54 2022] GET /identity/v3/role_assignments?user.id=cf4fb11822bf402a8fb5b4947411e894&scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3&role.id=d487417153354d90853ba306bd1edb13 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:54.429321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c4abc-d615-4c3b-b43b-342e479508ae None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:54.523264 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3141a2-476d-4f2e-813e-d1638d670f92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:54.527970 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3141a2-476d-4f2e-813e-d1638d670f92 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:54.597163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:54.614112 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:54.624315 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c4abc-d615-4c3b-b43b-342e479508ae None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:54.662359 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22961bcd-d2a1-4a07-810c-8b90407de371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:54.663385 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22961bcd-d2a1-4a07-810c-8b90407de371 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:54.663868 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22961bcd-d2a1-4a07-810c-8b90407de371 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:54.664476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22961bcd-d2a1-4a07-810c-8b90407de371 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:54.665421 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22961bcd-d2a1-4a07-810c-8b90407de371 None admin] RBAC: Adding query filter params (role.id=a836108447dc439e994611bc74d53d9c, scope.project.id=7d4cfda3691c4a4ab82dda7546659db6, user.id=970d0be8205f4a45a498452f92f1768c) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:54.665960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22961bcd-d2a1-4a07-810c-8b90407de371 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:54.667787 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22961bcd-d2a1-4a07-810c-8b90407de371 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:54.674451 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 109/205] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:23:54 2022] GET /identity/v3/role_assignments?user.id=970d0be8205f4a45a498452f92f1768c&scope.project.id=7d4cfda3691c4a4ab82dda7546659db6&role.id=a836108447dc439e994611bc74d53d9c => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:54.677962 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 97/206] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:23:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:23:54.684759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22961bcd-d2a1-4a07-810c-8b90407de371 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:54.803575 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22961bcd-d2a1-4a07-810c-8b90407de371 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:54.843332 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ec615-5ed9-437c-bff8-45990ca218bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:54.844332 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ec615-5ed9-437c-bff8-45990ca218bf None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:54.844785 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ec615-5ed9-437c-bff8-45990ca218bf None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:54.845224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ec615-5ed9-437c-bff8-45990ca218bf None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:54.846571 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ec615-5ed9-437c-bff8-45990ca218bf None admin] RBAC: Authorizing `identity:get_project(project_id=7d4cfda3691c4a4ab82dda7546659db6)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:54.848328 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ec615-5ed9-437c-bff8-45990ca218bf None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:54.850310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 110/207] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:23:54 2022] GET /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:54.856921 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3141a2-476d-4f2e-813e-d1638d670f92 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:54.982969 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3141a2-476d-4f2e-813e-d1638d670f92 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:55.027569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69daf6-7b91-4e51-a0c8-887e21fdf1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:55.028254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69daf6-7b91-4e51-a0c8-887e21fdf1cc None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:55.028637 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69daf6-7b91-4e51-a0c8-887e21fdf1cc None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:55.028914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69daf6-7b91-4e51-a0c8-887e21fdf1cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:55.029646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69daf6-7b91-4e51-a0c8-887e21fdf1cc None admin] RBAC: Adding query filter params (role.id=733cd39702ab4cbc860e34886dbb165f, scope.project.id=7d4cfda3691c4a4ab82dda7546659db6, user.id=970d0be8205f4a45a498452f92f1768c) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:55.029948 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69daf6-7b91-4e51-a0c8-887e21fdf1cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:55.031468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69daf6-7b91-4e51-a0c8-887e21fdf1cc None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:55.034574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:55.038288 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 98/208] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:23:54 2022] GET /identity/v3/role_assignments?user.id=970d0be8205f4a45a498452f92f1768c&scope.project.id=7d4cfda3691c4a4ab82dda7546659db6&role.id=733cd39702ab4cbc860e34886dbb165f => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:23:55.863192 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe65509-4d6d-406e-a4a0-398519d446ac None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:55.864275 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe65509-4d6d-406e-a4a0-398519d446ac None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:55.864787 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe65509-4d6d-406e-a4a0-398519d446ac None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:55.866344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 111/209] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:55.873044 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d9963-b83c-46c6-8647-c966bc568406 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:55.873567 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d9963-b83c-46c6-8647-c966bc568406 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:55.874049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d9963-b83c-46c6-8647-c966bc568406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:55.892804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-842d9963-b83c-46c6-8647-c966bc568406 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:55.894320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-842d9963-b83c-46c6-8647-c966bc568406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:55.966339 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 99/210] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:55.999066 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce746a-c6fb-48db-88df-63ce99ecb0b1 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:55.999391 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce746a-c6fb-48db-88df-63ce99ecb0b1 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:55.999608 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce746a-c6fb-48db-88df-63ce99ecb0b1 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:56.001145 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 112/211] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:56.006645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc35668-b926-449f-aac0-984a461e8287 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:56.007102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc35668-b926-449f-aac0-984a461e8287 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:56.007189 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc35668-b926-449f-aac0-984a461e8287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:56.025889 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-ccc35668-b926-449f-aac0-984a461e8287 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:56.027675 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-ccc35668-b926-449f-aac0-984a461e8287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:56.100535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 100/212] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:56.245014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce746a-c6fb-48db-88df-63ce99ecb0b1 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:56.274229 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:56.355602 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce746a-c6fb-48db-88df-63ce99ecb0b1 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:56.395379 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6df76-824f-4bc7-94e6-002d25b3e7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:56.396396 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6df76-824f-4bc7-94e6-002d25b3e7f9 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:56.396870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6df76-824f-4bc7-94e6-002d25b3e7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:56.397358 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6df76-824f-4bc7-94e6-002d25b3e7f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:56.410107 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc35668-b926-449f-aac0-984a461e8287 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:56.410478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6df76-824f-4bc7-94e6-002d25b3e7f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:56.417324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-54f6df76-824f-4bc7-94e6-002d25b3e7f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:23:56.417324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:56.417324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:56.417324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:56.417324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:56.417324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:56.417324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:56.417324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:56.417324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:56.417324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:56.417324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:56.417324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:56.417324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:56.417324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:56.417324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:56.417324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:56.418242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:56.418242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:56.418242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:56.418242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:56.418242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:56.418242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:56.418242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:56.418242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:56.418242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:56.418242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:56.418242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:56.418242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:56.418242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:56.418242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:56.418242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:56.418242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:56.419080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:56.419080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:56.419080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:56.419080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:56.419080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:56.419080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:56.419080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:56.419080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:56.419080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:56.419080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:56.419080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:23:56.419080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:23:56.421494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 113/213] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:23:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:23:56.427276 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6df76-824f-4bc7-94e6-002d25b3e7f9 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:56.492915 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:56.543635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:56.549453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc35668-b926-449f-aac0-984a461e8287 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:56.563962 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6df76-824f-4bc7-94e6-002d25b3e7f9 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:56.596080 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d529a4-dd6a-4e63-a9fe-42b85e6bb5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:56.597223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d529a4-dd6a-4e63-a9fe-42b85e6bb5c0 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:56.597981 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d529a4-dd6a-4e63-a9fe-42b85e6bb5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:56.598697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d529a4-dd6a-4e63-a9fe-42b85e6bb5c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:56.600322 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d529a4-dd6a-4e63-a9fe-42b85e6bb5c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:56.602577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d529a4-dd6a-4e63-a9fe-42b85e6bb5c0 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:56.605708 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 101/214] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:23:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:56.611323 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d529a4-dd6a-4e63-a9fe-42b85e6bb5c0 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:56.613604 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b344e6-7cdf-41cd-bc12-072d769b9ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:56.614221 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b344e6-7cdf-41cd-bc12-072d769b9ba1 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:56.614498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b344e6-7cdf-41cd-bc12-072d769b9ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:56.614763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b344e6-7cdf-41cd-bc12-072d769b9ba1 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:56.615538 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b344e6-7cdf-41cd-bc12-072d769b9ba1 None admin] RBAC: Adding query filter params (name=member) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:56.615836 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b344e6-7cdf-41cd-bc12-072d769b9ba1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:56.617484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b344e6-7cdf-41cd-bc12-072d769b9ba1 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:56.631993 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 114/215] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:23:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:23:56.633296 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b344e6-7cdf-41cd-bc12-072d769b9ba1 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:56.728258 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d529a4-dd6a-4e63-a9fe-42b85e6bb5c0 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:56.740040 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b344e6-7cdf-41cd-bc12-072d769b9ba1 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:56.759333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:56.769240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d963d-fdde-4995-bbc4-b7ee23fb5da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:56.769516 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d963d-fdde-4995-bbc4-b7ee23fb5da3 None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:56.769814 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d963d-fdde-4995-bbc4-b7ee23fb5da3 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:56.770149 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d963d-fdde-4995-bbc4-b7ee23fb5da3 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:56.770800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d963d-fdde-4995-bbc4-b7ee23fb5da3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:56.772259 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d963d-fdde-4995-bbc4-b7ee23fb5da3 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:56.776597 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c115e7-5ef1-4cf6-9a23-db1f49a98c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:56.778022 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c115e7-5ef1-4cf6-9a23-db1f49a98c3a None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:56.778321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c115e7-5ef1-4cf6-9a23-db1f49a98c3a None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:56.778645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c115e7-5ef1-4cf6-9a23-db1f49a98c3a None admin] PATH_INFO: `/v3/users/5489097abadb4c039a0e4283bf7db21e` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:56.795074 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c115e7-5ef1-4cf6-9a23-db1f49a98c3a None admin] RBAC: Authorizing `identity:get_user(user_id=5489097abadb4c039a0e4283bf7db21e)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:56.798767 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c115e7-5ef1-4cf6-9a23-db1f49a98c3a None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:56.798767 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 115/216] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:56 2022] GET /identity/v3/users/5489097abadb4c039a0e4283bf7db21e => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:56.800315 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8efb5-9d62-4c0e-ba12-a6b45080e784 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:56.800605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8efb5-9d62-4c0e-ba12-a6b45080e784 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:56.800873 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8efb5-9d62-4c0e-ba12-a6b45080e784 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:56.802235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 116/217] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:56.803947 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8efb5-9d62-4c0e-ba12-a6b45080e784 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:56.809952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:56.899394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 102/218] 10.222.0.59 () {60 vars in 1132 bytes} [Mon Aug 29 22:23:56 2022] POST /identity/v3/users => generated 297 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:23:56.907300 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ed853-b979-46eb-a5a4-5c2ae8043544 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:56.907300 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ed853-b979-46eb-a5a4-5c2ae8043544 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:56.907300 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ed853-b979-46eb-a5a4-5c2ae8043544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:56.930331 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8efb5-9d62-4c0e-ba12-a6b45080e784 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:56.935641 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-a79ed853-b979-46eb-a5a4-5c2ae8043544 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:56.936949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a79ed853-b979-46eb-a5a4-5c2ae8043544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:56.981800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038c413-b1c7-4b4a-ba5c-604a2c2f4947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:56.982761 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038c413-b1c7-4b4a-ba5c-604a2c2f4947 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:56.983095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038c413-b1c7-4b4a-ba5c-604a2c2f4947 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:56.983375 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038c413-b1c7-4b4a-ba5c-604a2c2f4947 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:57.032694 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 103/219] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:57.037804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb5138-91c0-4455-a9f6-aa636046374b None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:57.038085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb5138-91c0-4455-a9f6-aa636046374b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:57.038354 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb5138-91c0-4455-a9f6-aa636046374b None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:57.039775 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 104/220] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:57.046585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dbda7-0e27-4f56-b0df-6dad04be350d None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:57.046767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dbda7-0e27-4f56-b0df-6dad04be350d None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:57.047050 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dbda7-0e27-4f56-b0df-6dad04be350d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:57.066394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-245dbda7-0e27-4f56-b0df-6dad04be350d None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:57.067605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-245dbda7-0e27-4f56-b0df-6dad04be350d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:57.075171 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038c413-b1c7-4b4a-ba5c-604a2c2f4947 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:57.077020 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038c413-b1c7-4b4a-ba5c-604a2c2f4947 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:57.079995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:57.158347 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 105/221] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:23:57.208174 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 117/222] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:23:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 22:23:57.211249 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245dbda7-0e27-4f56-b0df-6dad04be350d None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:57.271717 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:57.321817 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038c413-b1c7-4b4a-ba5c-604a2c2f4947 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:57.341043 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245dbda7-0e27-4f56-b0df-6dad04be350d None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:57.379724 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67a297-531c-4265-8ce5-c880929f122b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:57.380417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67a297-531c-4265-8ce5-c880929f122b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:57.380703 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67a297-531c-4265-8ce5-c880929f122b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:57.380974 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67a297-531c-4265-8ce5-c880929f122b None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:57.382188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b67a297-531c-4265-8ce5-c880929f122b None admin] RBAC: Authorizing `identity:get_project(project_id=7d4cfda3691c4a4ab82dda7546659db6)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:57.383780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b67a297-531c-4265-8ce5-c880929f122b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:57.385463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 106/223] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:23:57 2022] GET /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:57.391700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67a297-531c-4265-8ce5-c880929f122b None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:57.456714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038c413-b1c7-4b4a-ba5c-604a2c2f4947 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:57.460499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:57.497556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67a297-531c-4265-8ce5-c880929f122b None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:57.509630 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f43d3-4f96-4453-9f5e-4f00a8834152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:57.510259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f43d3-4f96-4453-9f5e-4f00a8834152 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:57.510528 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f43d3-4f96-4453-9f5e-4f00a8834152 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:57.510789 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f43d3-4f96-4453-9f5e-4f00a8834152 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:57.516525 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:57.522518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f43d3-4f96-4453-9f5e-4f00a8834152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:57.527645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-f68f43d3-4f96-4453-9f5e-4f00a8834152 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:23:57.527645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:57.527645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:57.527645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:57.527645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:57.527645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:57.527645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:57.527645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:57.527645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:57.527645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:57.527645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:57.527645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:57.527645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:57.527645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:57.527645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:57.527645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:57.528588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:57.528588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:57.528588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:57.528588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:57.528588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:57.528588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:57.528588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:57.528588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:57.528588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:57.528588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:57.528588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:57.528588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:57.528588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:57.528588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:57.528588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:57.528588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:57.529382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:57.529382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:57.529382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:57.529382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:57.529382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:57.529382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:57.529382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:57.529382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:57.529382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:57.529382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:57.529382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:23:57.529382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:23:57.529382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 118/224] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:23:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:23:57.530016 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f43d3-4f96-4453-9f5e-4f00a8834152 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:57.537897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa25142-5e28-4ecb-aad8-7a6be51dbfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:57.538559 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa25142-5e28-4ecb-aad8-7a6be51dbfd3 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:57.543261 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa25142-5e28-4ecb-aad8-7a6be51dbfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:57.543261 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa25142-5e28-4ecb-aad8-7a6be51dbfd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:57.543261 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa25142-5e28-4ecb-aad8-7a6be51dbfd3 None admin] RBAC: Adding query filter params (role.id=454bd427dfc04030a4ec2b0e8a1ed237, scope.project.id=7d4cfda3691c4a4ab82dda7546659db6, user.id=5489097abadb4c039a0e4283bf7db21e) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:57.543261 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa25142-5e28-4ecb-aad8-7a6be51dbfd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:57.543261 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa25142-5e28-4ecb-aad8-7a6be51dbfd3 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:57.550171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 107/225] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:23:57 2022] GET /identity/v3/role_assignments?user.id=5489097abadb4c039a0e4283bf7db21e&scope.project.id=7d4cfda3691c4a4ab82dda7546659db6&role.id=454bd427dfc04030a4ec2b0e8a1ed237 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:23:57.551477 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa25142-5e28-4ecb-aad8-7a6be51dbfd3 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:57.682785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa25142-5e28-4ecb-aad8-7a6be51dbfd3 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:57.723346 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f43d3-4f96-4453-9f5e-4f00a8834152 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:57.725405 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0a99c8-e464-4e6b-a8ee-30c3a2e07c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:57.727075 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a99c8-e464-4e6b-a8ee-30c3a2e07c7c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:57.727356 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a99c8-e464-4e6b-a8ee-30c3a2e07c7c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:57.727628 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a99c8-e464-4e6b-a8ee-30c3a2e07c7c None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:57.728343 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0a99c8-e464-4e6b-a8ee-30c3a2e07c7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:57.728712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0a99c8-e464-4e6b-a8ee-30c3a2e07c7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:57.730749 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0a99c8-e464-4e6b-a8ee-30c3a2e07c7c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:57.740874 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 108/226] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:23:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:57.748968 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0a99c8-e464-4e6b-a8ee-30c3a2e07c7c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:57.783998 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6772bbc-11bb-4d80-a2c2-ef735f2ae7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:57.784711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6772bbc-11bb-4d80-a2c2-ef735f2ae7e4 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:57.784987 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6772bbc-11bb-4d80-a2c2-ef735f2ae7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:57.785253 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6772bbc-11bb-4d80-a2c2-ef735f2ae7e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:57.802122 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6772bbc-11bb-4d80-a2c2-ef735f2ae7e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:23:57.807399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-c6772bbc-11bb-4d80-a2c2-ef735f2ae7e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:23:57.807399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:23:57.807399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:23:57.807399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:23:57.807399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:23:57.807399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:23:57.807399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:23:57.807399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:23:57.807399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:23:57.807399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:23:57.807399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:23:57.807399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:23:57.807399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:23:57.807399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:23:57.807399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:23:57.807399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:23:57.809041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:23:57.809041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:23:57.809041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:23:57.809041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:23:57.809041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:23:57.809041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:23:57.809041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:23:57.809041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:23:57.809041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:23:57.809041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:23:57.809041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:23:57.809041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:23:57.809041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:23:57.809041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:23:57.809041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:23:57.809041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:23:57.809811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:23:57.809811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:23:57.809811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:23:57.809811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:23:57.809811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:23:57.809811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:23:57.809811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:23:57.809811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:23:57.809811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:23:57.809811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:23:57.809811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:23:57.809811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:23:57.809811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 119/227] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:23:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:23:57.815457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6772bbc-11bb-4d80-a2c2-ef735f2ae7e4 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:57.817896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:57.871549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:57.897092 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0a99c8-e464-4e6b-a8ee-30c3a2e07c7c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:57.939782 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a851b9bb-4d46-4aeb-bb06-233584d9142d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:57.940597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851b9bb-4d46-4aeb-bb06-233584d9142d None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:57.940932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851b9bb-4d46-4aeb-bb06-233584d9142d None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:57.941253 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851b9bb-4d46-4aeb-bb06-233584d9142d None admin] PATH_INFO: `/v3/users/970d0be8205f4a45a498452f92f1768c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:57.943194 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a851b9bb-4d46-4aeb-bb06-233584d9142d None admin] RBAC: Authorizing `identity:get_user(user_id=970d0be8205f4a45a498452f92f1768c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:57.944835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a851b9bb-4d46-4aeb-bb06-233584d9142d None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:57.947660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 109/228] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:57 2022] GET /identity/v3/users/970d0be8205f4a45a498452f92f1768c => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:23:57.953254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a851b9bb-4d46-4aeb-bb06-233584d9142d None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:58.032570 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6772bbc-11bb-4d80-a2c2-ef735f2ae7e4 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:58.113845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bae2f2-fa35-4c05-a491-6bd16ad70765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:58.114474 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bae2f2-fa35-4c05-a491-6bd16ad70765 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:58.114750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bae2f2-fa35-4c05-a491-6bd16ad70765 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:58.115014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bae2f2-fa35-4c05-a491-6bd16ad70765 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:58.115686 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bae2f2-fa35-4c05-a491-6bd16ad70765 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:23:58.115982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bae2f2-fa35-4c05-a491-6bd16ad70765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:58.117398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bae2f2-fa35-4c05-a491-6bd16ad70765 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:58.135485 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 120/229] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:23:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:23:58.143159 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bae2f2-fa35-4c05-a491-6bd16ad70765 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:58.160963 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a851b9bb-4d46-4aeb-bb06-233584d9142d None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:58.168915 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:58.211565 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae519140-f67b-4465-a028-cf52839a8341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:58.211565 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae519140-f67b-4465-a028-cf52839a8341 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:58.211565 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae519140-f67b-4465-a028-cf52839a8341 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:58.211565 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae519140-f67b-4465-a028-cf52839a8341 None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:58.213343 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae519140-f67b-4465-a028-cf52839a8341 None admin] RBAC: Authorizing `identity:get_project(project_id=7d4cfda3691c4a4ab82dda7546659db6)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:58.215834 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae519140-f67b-4465-a028-cf52839a8341 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:58.218296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 110/230] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:23:57 2022] GET /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6 => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:58.224954 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae519140-f67b-4465-a028-cf52839a8341 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:58.330014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:58.363107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bae2f2-fa35-4c05-a491-6bd16ad70765 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:58.437750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae519140-f67b-4465-a028-cf52839a8341 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:58.438473 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9162a-2120-4789-9ebb-11f2e6717c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:58.439316 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9162a-2120-4789-9ebb-11f2e6717c1d None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:58.441249 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9162a-2120-4789-9ebb-11f2e6717c1d None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:58.441706 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9162a-2120-4789-9ebb-11f2e6717c1d None admin] PATH_INFO: `/v3/users/970d0be8205f4a45a498452f92f1768c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:58.443158 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc9162a-2120-4789-9ebb-11f2e6717c1d None admin] RBAC: Authorizing `identity:get_user(user_id=970d0be8205f4a45a498452f92f1768c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:58.444974 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc9162a-2120-4789-9ebb-11f2e6717c1d None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:58.447595 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 121/231] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:23:58 2022] GET /identity/v3/users/970d0be8205f4a45a498452f92f1768c => generated 283 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:23:58.458060 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9162a-2120-4789-9ebb-11f2e6717c1d None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:58.536445 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8277723-25a9-4efd-a063-c892753ca35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:58.537051 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8277723-25a9-4efd-a063-c892753ca35d None admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:58.537330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8277723-25a9-4efd-a063-c892753ca35d None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:58.537602 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8277723-25a9-4efd-a063-c892753ca35d None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6/users/970d0be8205f4a45a498452f92f1768c/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:58.541539 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8277723-25a9-4efd-a063-c892753ca35d None admin] RBAC: Authorizing `identity:create_grant(project_id=7d4cfda3691c4a4ab82dda7546659db6, user_id=970d0be8205f4a45a498452f92f1768c, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:58.545008 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8277723-25a9-4efd-a063-c892753ca35d None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:58.589938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9162a-2120-4789-9ebb-11f2e6717c1d None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:58.626753 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c972dd2-61f6-4b9b-8ae4-3a1340121ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:58.627499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c972dd2-61f6-4b9b-8ae4-3a1340121ddb None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:58.627785 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c972dd2-61f6-4b9b-8ae4-3a1340121ddb None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:58.628060 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c972dd2-61f6-4b9b-8ae4-3a1340121ddb None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:58.629208 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c972dd2-61f6-4b9b-8ae4-3a1340121ddb None admin] RBAC: Authorizing `identity:get_project(project_id=7d4cfda3691c4a4ab82dda7546659db6)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:58.630729 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c972dd2-61f6-4b9b-8ae4-3a1340121ddb None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:58.632763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 122/232] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:23:58 2022] GET /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:23:58.638434 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c972dd2-61f6-4b9b-8ae4-3a1340121ddb None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:23:58.648090 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 111/233] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:23:58 2022] PUT /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6/users/970d0be8205f4a45a498452f92f1768c/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:23:58.688250 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:58.797522 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c972dd2-61f6-4b9b-8ae4-3a1340121ddb None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:23:58.844447 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adf457-5022-4e66-abb1-4395aa89062e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:23:58.845128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adf457-5022-4e66-abb1-4395aa89062e None admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:58.845459 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adf457-5022-4e66-abb1-4395aa89062e None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:58.845735 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adf457-5022-4e66-abb1-4395aa89062e None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6/users/970d0be8205f4a45a498452f92f1768c/roles/733cd39702ab4cbc860e34886dbb165f` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:58.848035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96adf457-5022-4e66-abb1-4395aa89062e None admin] RBAC: Authorizing `identity:create_grant(project_id=7d4cfda3691c4a4ab82dda7546659db6, user_id=970d0be8205f4a45a498452f92f1768c, role_id=733cd39702ab4cbc860e34886dbb165f)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:23:58.849543 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96adf457-5022-4e66-abb1-4395aa89062e None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:23:58.962765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 123/234] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:23:58 2022] PUT /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6/users/970d0be8205f4a45a498452f92f1768c/roles/733cd39702ab4cbc860e34886dbb165f => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:23:59.470626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97129a43-0022-400d-b0d4-ea7a56280cb9 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:59.471144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97129a43-0022-400d-b0d4-ea7a56280cb9 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:59.471144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97129a43-0022-400d-b0d4-ea7a56280cb9 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:59.472909 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 112/235] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:23:59.478549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41bca6-ac50-4e8b-b245-81b1352d2885 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:59.478965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41bca6-ac50-4e8b-b245-81b1352d2885 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:59.479171 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41bca6-ac50-4e8b-b245-81b1352d2885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:59.508727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-eb41bca6-ac50-4e8b-b245-81b1352d2885 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:59.509932 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-eb41bca6-ac50-4e8b-b245-81b1352d2885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:59.560092 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452a424-eb4f-4383-9805-5ea0db30af10 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:59.560417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452a424-eb4f-4383-9805-5ea0db30af10 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:59.560664 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452a424-eb4f-4383-9805-5ea0db30af10 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:59.562122 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 113/236] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:23:59.575113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1653f-53f4-42c7-aacb-3cbc8ae1eaa7 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:23:59.575409 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1653f-53f4-42c7-aacb-3cbc8ae1eaa7 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:23:59.575649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1653f-53f4-42c7-aacb-3cbc8ae1eaa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:23:59.600776 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-98c1653f-53f4-42c7-aacb-3cbc8ae1eaa7 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:23:59.602003 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-98c1653f-53f4-42c7-aacb-3cbc8ae1eaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:23:59.637307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 124/237] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:23:59.659184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:23:59.718455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 114/238] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:24:00.043983 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41bca6-ac50-4e8b-b245-81b1352d2885 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:00.061403 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:00.071729 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1653f-53f4-42c7-aacb-3cbc8ae1eaa7 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:00.180416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41bca6-ac50-4e8b-b245-81b1352d2885 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:00.192297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1653f-53f4-42c7-aacb-3cbc8ae1eaa7 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:00.218880 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b83f0d-7cde-468e-8e24-01a50c0c3f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:00.219819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b83f0d-7cde-468e-8e24-01a50c0c3f57 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:00.220301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b83f0d-7cde-468e-8e24-01a50c0c3f57 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:00.220730 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b83f0d-7cde-468e-8e24-01a50c0c3f57 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:00.236440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b83f0d-7cde-468e-8e24-01a50c0c3f57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:00.236693 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7be8f-3219-4fba-9fbb-84237140cda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:00.237450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7be8f-3219-4fba-9fbb-84237140cda7 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:00.237818 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7be8f-3219-4fba-9fbb-84237140cda7 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:00.238162 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7be8f-3219-4fba-9fbb-84237140cda7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:00.239339 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7be8f-3219-4fba-9fbb-84237140cda7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:00.240901 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7be8f-3219-4fba-9fbb-84237140cda7 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:00.247657 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 115/239] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:00.247657 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68dc55-c1c2-47c8-9cd6-bc4b9cc70169 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:00.247657 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68dc55-c1c2-47c8-9cd6-bc4b9cc70169 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:00.247657 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68dc55-c1c2-47c8-9cd6-bc4b9cc70169 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:00.247657 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 116/240] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:00.248607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-81b83f0d-7cde-468e-8e24-01a50c0c3f57 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:24:00.248607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:00.248607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:00.248607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:00.248607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:00.248607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:00.248607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:00.248607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:00.248607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:00.248607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:00.248607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:00.248607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:00.248607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:00.248607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:00.248607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:00.248607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:00.249439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:00.249439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:00.249439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:00.249439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:00.249439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:00.249439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:00.249439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:00.249439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:00.249439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:00.249439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:00.249439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:00.249439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:00.249439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:00.249439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:00.249439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:00.249439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:00.250224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:00.250224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:00.250224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:00.250224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:00.250224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:00.250224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:00.250224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:00.250224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:00.250224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:00.250224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:00.250224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:24:00.250224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:00.250224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 125/241] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:00.251308 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68dc55-c1c2-47c8-9cd6-bc4b9cc70169 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:00.256833 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b83f0d-7cde-468e-8e24-01a50c0c3f57 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:00.290331 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:00.351718 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:00.386344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68dc55-c1c2-47c8-9cd6-bc4b9cc70169 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:00.397555 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b83f0d-7cde-468e-8e24-01a50c0c3f57 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:00.426215 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ee923-d49a-4ca9-875d-2b5ee9614f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:00.427290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ee923-d49a-4ca9-875d-2b5ee9614f6b None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:00.427852 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ee923-d49a-4ca9-875d-2b5ee9614f6b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:00.428400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ee923-d49a-4ca9-875d-2b5ee9614f6b None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:00.429291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ee923-d49a-4ca9-875d-2b5ee9614f6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:00.431226 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ee923-d49a-4ca9-875d-2b5ee9614f6b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:00.435917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae52af5-9912-44d3-9f86-63ab30770972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:00.436561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae52af5-9912-44d3-9f86-63ab30770972 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:00.436843 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae52af5-9912-44d3-9f86-63ab30770972 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:00.437116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae52af5-9912-44d3-9f86-63ab30770972 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:00.437783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae52af5-9912-44d3-9f86-63ab30770972 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:00.438084 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae52af5-9912-44d3-9f86-63ab30770972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:00.439475 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae52af5-9912-44d3-9f86-63ab30770972 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:00.450394 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 126/242] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:24:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:00.451106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa273f0e-3f54-4ca0-8a2e-68300a165147 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:00.451447 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa273f0e-3f54-4ca0-8a2e-68300a165147 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:00.451713 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa273f0e-3f54-4ca0-8a2e-68300a165147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:00.472506 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-fa273f0e-3f54-4ca0-8a2e-68300a165147 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:00.473705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-fa273f0e-3f54-4ca0-8a2e-68300a165147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:00.548543 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 127/244] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:00.552849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 117/244] 10.222.0.59 () {60 vars in 1132 bytes} [Mon Aug 29 22:24:00 2022] POST /identity/v3/users => generated 293 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:24:00.552849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ee923-d49a-4ca9-875d-2b5ee9614f6b None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:00.664972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:00.688891 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ee923-d49a-4ca9-875d-2b5ee9614f6b None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:00.736543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452aa0c5-b5d4-4f9f-a72c-f9b154f8a0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:00.737176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452aa0c5-b5d4-4f9f-a72c-f9b154f8a0dd None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:00.737451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452aa0c5-b5d4-4f9f-a72c-f9b154f8a0dd None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:00.737717 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452aa0c5-b5d4-4f9f-a72c-f9b154f8a0dd None admin] PATH_INFO: `/v3/users/2743aeab95ff4ba790b49339ce90d5ce` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:00.759928 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452aa0c5-b5d4-4f9f-a72c-f9b154f8a0dd None admin] RBAC: Authorizing `identity:get_user(user_id=2743aeab95ff4ba790b49339ce90d5ce)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:00.762598 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452aa0c5-b5d4-4f9f-a72c-f9b154f8a0dd None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:00.764656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 118/245] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:24:00 2022] GET /identity/v3/users/2743aeab95ff4ba790b49339ce90d5ce => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:00.770311 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa273f0e-3f54-4ca0-8a2e-68300a165147 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:00.848811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:00.863718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452aa0c5-b5d4-4f9f-a72c-f9b154f8a0dd None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:00.885124 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa273f0e-3f54-4ca0-8a2e-68300a165147 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:00.932553 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a7507-b9a3-448d-af52-e90728acb72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:00.932553 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a7507-b9a3-448d-af52-e90728acb72a None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:00.932553 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a7507-b9a3-448d-af52-e90728acb72a None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:00.932553 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a7507-b9a3-448d-af52-e90728acb72a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:01.006486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a7507-b9a3-448d-af52-e90728acb72a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:01.007309 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452aa0c5-b5d4-4f9f-a72c-f9b154f8a0dd None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:01.007865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a7507-b9a3-448d-af52-e90728acb72a None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:01.049045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e50b6-455e-4e0e-a5fb-60347bfd405a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:01.050142 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e50b6-455e-4e0e-a5fb-60347bfd405a None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:01.050621 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e50b6-455e-4e0e-a5fb-60347bfd405a None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:01.051089 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e50b6-455e-4e0e-a5fb-60347bfd405a None admin] PATH_INFO: `/v3/roles/member` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:01.064212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8e50b6-455e-4e0e-a5fb-60347bfd405a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:01.069719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-6c8e50b6-455e-4e0e-a5fb-60347bfd405a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:01.069719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:01.069719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:01.069719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:01.069719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:01.069719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:01.069719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:01.069719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:01.069719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:01.069719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:01.069719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:01.069719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:01.069719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:01.069719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:01.069719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:01.069719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:01.070610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:01.070610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:01.070610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:01.070610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:01.070610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:01.070610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:01.070610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:01.070610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:01.070610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:01.070610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:01.070610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:01.070610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:01.070610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:01.070610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:01.070610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:01.070610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:01.071403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:01.071403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:01.071403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:01.071403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:01.071403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:01.071403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:01.071403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:01.071403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:01.071403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:01.071403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:01.071403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:01.071403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:01.074188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 119/246] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:01.076235 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670890a4-890e-42e3-9a18-afdd062be482 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:01.076692 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670890a4-890e-42e3-9a18-afdd062be482 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:01.077114 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670890a4-890e-42e3-9a18-afdd062be482 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:01.078554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 120/247] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:01.079382 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670890a4-890e-42e3-9a18-afdd062be482 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:01.082534 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:01.090245 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:01.128226 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 128/248] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 22:24:01.129719 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd6e43-35f7-4e6f-aacf-198186b3c97b None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:01.130139 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd6e43-35f7-4e6f-aacf-198186b3c97b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:01.130541 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd6e43-35f7-4e6f-aacf-198186b3c97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:01.150433 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-93bd6e43-35f7-4e6f-aacf-198186b3c97b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:01.152529 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-93bd6e43-35f7-4e6f-aacf-198186b3c97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:01.214700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670890a4-890e-42e3-9a18-afdd062be482 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:01.238837 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 129/249] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:01.239836 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd6e43-35f7-4e6f-aacf-198186b3c97b None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:01.253810 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f739c7c-fb0a-4fac-ac95-bece713f82fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:01.254717 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f739c7c-fb0a-4fac-ac95-bece713f82fb None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:01.255152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f739c7c-fb0a-4fac-ac95-bece713f82fb None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:01.255705 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f739c7c-fb0a-4fac-ac95-bece713f82fb None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:01.256715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f739c7c-fb0a-4fac-ac95-bece713f82fb None admin] RBAC: Adding query filter params (name=member) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:01.257262 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f739c7c-fb0a-4fac-ac95-bece713f82fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:01.258970 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f739c7c-fb0a-4fac-ac95-bece713f82fb None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:01.269368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 121/250] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:24:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:01.277725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f739c7c-fb0a-4fac-ac95-bece713f82fb None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:01.383646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:01.392156 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd6e43-35f7-4e6f-aacf-198186b3c97b None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:01.429372 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e5dad-f52d-48d8-a9c5-a7095b6add88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:01.429916 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:01.430026 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e5dad-f52d-48d8-a9c5-a7095b6add88 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:01.430268 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e5dad-f52d-48d8-a9c5-a7095b6add88 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:01.430532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e5dad-f52d-48d8-a9c5-a7095b6add88 None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:01.431786 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e5dad-f52d-48d8-a9c5-a7095b6add88 None admin] RBAC: Authorizing `identity:get_project(project_id=7d4cfda3691c4a4ab82dda7546659db6)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:01.433408 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e5dad-f52d-48d8-a9c5-a7095b6add88 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:01.435348 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 130/251] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:01 2022] GET /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:01.441417 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e5dad-f52d-48d8-a9c5-a7095b6add88 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:01.498228 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f739c7c-fb0a-4fac-ac95-bece713f82fb None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:01.556248 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e5dad-f52d-48d8-a9c5-a7095b6add88 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:01.556755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b40651-e0d5-47d8-aa66-58a64261fc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:01.556755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b40651-e0d5-47d8-aa66-58a64261fc3c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:01.556755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b40651-e0d5-47d8-aa66-58a64261fc3c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:01.557213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b40651-e0d5-47d8-aa66-58a64261fc3c None admin] PATH_INFO: `/v3/users/5489097abadb4c039a0e4283bf7db21e` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:01.559196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b40651-e0d5-47d8-aa66-58a64261fc3c None admin] RBAC: Authorizing `identity:get_user(user_id=5489097abadb4c039a0e4283bf7db21e)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:01.565202 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b40651-e0d5-47d8-aa66-58a64261fc3c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:01.567098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 122/252] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:24:01 2022] GET /identity/v3/users/5489097abadb4c039a0e4283bf7db21e => generated 297 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:01.568037 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b797a4f-d82a-468e-8b11-fd3094358985 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:01.568341 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b797a4f-d82a-468e-8b11-fd3094358985 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:01.568608 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b797a4f-d82a-468e-8b11-fd3094358985 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:01.572077 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 123/253] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:01.575284 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b797a4f-d82a-468e-8b11-fd3094358985 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:01.616704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb1d0b-739c-42eb-94a8-5cae6191f723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:01.617322 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb1d0b-739c-42eb-94a8-5cae6191f723 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:01.617592 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb1d0b-739c-42eb-94a8-5cae6191f723 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:01.617853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb1d0b-739c-42eb-94a8-5cae6191f723 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:01.618574 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb1d0b-739c-42eb-94a8-5cae6191f723 None admin] RBAC: Adding query filter params (role.id=d487417153354d90853ba306bd1edb13, scope.project.id=7d4cfda3691c4a4ab82dda7546659db6, user.id=2743aeab95ff4ba790b49339ce90d5ce) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:01.618872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb1d0b-739c-42eb-94a8-5cae6191f723 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:01.620357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb1d0b-739c-42eb-94a8-5cae6191f723 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:01.630322 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 131/254] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:24:01 2022] GET /identity/v3/role_assignments?user.id=2743aeab95ff4ba790b49339ce90d5ce&scope.project.id=7d4cfda3691c4a4ab82dda7546659db6&role.id=d487417153354d90853ba306bd1edb13 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:01.631534 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2635901-26a8-4f61-8ebd-7402955ec1c1 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:01.631808 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2635901-26a8-4f61-8ebd-7402955ec1c1 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:01.632069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2635901-26a8-4f61-8ebd-7402955ec1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:01.693383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-f2635901-26a8-4f61-8ebd-7402955ec1c1 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:01.694846 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f2635901-26a8-4f61-8ebd-7402955ec1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:01.747977 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b797a4f-d82a-468e-8b11-fd3094358985 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:01.764518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:01.817509 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6796f-388f-4266-ba48-075c413d17c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:01.818074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6796f-388f-4266-ba48-075c413d17c7 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:01.818371 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6796f-388f-4266-ba48-075c413d17c7 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:01.818644 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6796f-388f-4266-ba48-075c413d17c7 None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:01.820231 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6796f-388f-4266-ba48-075c413d17c7 None admin] RBAC: Authorizing `identity:get_project(project_id=7d4cfda3691c4a4ab82dda7546659db6)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:01.821956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6796f-388f-4266-ba48-075c413d17c7 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:01.824790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 124/255] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:01 2022] GET /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:01.825635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6796f-388f-4266-ba48-075c413d17c7 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:01.829669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:01.840355 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 132/256] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:01.849152 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2635901-26a8-4f61-8ebd-7402955ec1c1 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:01.962878 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6796f-388f-4266-ba48-075c413d17c7 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:02.003626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be4a8d-f0a8-4976-bca5-4d496d331147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:02.004291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be4a8d-f0a8-4976-bca5-4d496d331147 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:02.004625 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be4a8d-f0a8-4976-bca5-4d496d331147 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:02.004900 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be4a8d-f0a8-4976-bca5-4d496d331147 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:02.018302 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be4a8d-f0a8-4976-bca5-4d496d331147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:02.024248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-e4be4a8d-f0a8-4976-bca5-4d496d331147 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:24:02.024248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:02.024248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:02.024248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:02.024248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:02.024248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:02.024248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:02.024248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:02.024248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:02.024248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:02.024248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:02.024248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:02.024248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:02.024248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:02.024248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:02.024248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:02.025154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:02.025154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:02.025154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:02.025154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:02.025154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:02.025154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:02.025154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:02.025154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:02.025154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:02.025154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:02.025154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:02.025154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:02.025154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:02.025154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:02.025154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:02.025154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:02.025957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:02.025957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:02.025957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:02.025957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:02.025957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:02.025957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:02.025957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:02.025957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:02.025957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:02.025957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:02.025957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:24:02.025957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:02.025957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 125/257] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:24:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:02.034729 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be4a8d-f0a8-4976-bca5-4d496d331147 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:02.112648 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:02.135011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2635901-26a8-4f61-8ebd-7402955ec1c1 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:02.148928 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be4a8d-f0a8-4976-bca5-4d496d331147 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:02.188843 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156da0de-a5d3-4ff8-b43f-34ad1153f1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:02.189745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156da0de-a5d3-4ff8-b43f-34ad1153f1ee None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:02.190034 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156da0de-a5d3-4ff8-b43f-34ad1153f1ee None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:02.190312 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156da0de-a5d3-4ff8-b43f-34ad1153f1ee None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:02.197045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156da0de-a5d3-4ff8-b43f-34ad1153f1ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:02.197045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156da0de-a5d3-4ff8-b43f-34ad1153f1ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:02.197045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156da0de-a5d3-4ff8-b43f-34ad1153f1ee None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:02.202352 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 126/258] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:02.210923 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156da0de-a5d3-4ff8-b43f-34ad1153f1ee None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:02.231008 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ee64f-13e0-4369-a0de-4defcf9a0c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:02.231693 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ee64f-13e0-4369-a0de-4defcf9a0c33 None admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:02.231971 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ee64f-13e0-4369-a0de-4defcf9a0c33 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:02.232277 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ee64f-13e0-4369-a0de-4defcf9a0c33 None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6/users/5489097abadb4c039a0e4283bf7db21e/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:02.236884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ee64f-13e0-4369-a0de-4defcf9a0c33 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d4cfda3691c4a4ab82dda7546659db6, user_id=5489097abadb4c039a0e4283bf7db21e, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:02.238335 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ee64f-13e0-4369-a0de-4defcf9a0c33 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:02.317889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 133/259] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:24:01 2022] PUT /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6/users/5489097abadb4c039a0e4283bf7db21e/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:24:02.318606 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ee64f-13e0-4369-a0de-4defcf9a0c33 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:02.342956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156da0de-a5d3-4ff8-b43f-34ad1153f1ee None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:02.359176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:02.393122 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:02.412671 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49c3f3-40c7-40a8-9182-fa749c639ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:02.413311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49c3f3-40c7-40a8-9182-fa749c639ab2 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:02.413587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49c3f3-40c7-40a8-9182-fa749c639ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:02.413856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49c3f3-40c7-40a8-9182-fa749c639ab2 None admin] PATH_INFO: `/v3/users/970d0be8205f4a45a498452f92f1768c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:02.415441 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49c3f3-40c7-40a8-9182-fa749c639ab2 None admin] RBAC: Authorizing `identity:get_user(user_id=970d0be8205f4a45a498452f92f1768c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:02.417044 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49c3f3-40c7-40a8-9182-fa749c639ab2 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:02.419467 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 127/260] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:24:02 2022] GET /identity/v3/users/970d0be8205f4a45a498452f92f1768c => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:02.425367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49c3f3-40c7-40a8-9182-fa749c639ab2 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:02.486062 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ee64f-13e0-4369-a0de-4defcf9a0c33 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:02.541210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d76c29-25c4-45e3-bde7-2ffe719e0e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:02.541864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d76c29-25c4-45e3-bde7-2ffe719e0e7c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:02.542753 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d76c29-25c4-45e3-bde7-2ffe719e0e7c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:02.543114 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d76c29-25c4-45e3-bde7-2ffe719e0e7c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:02.556905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d76c29-25c4-45e3-bde7-2ffe719e0e7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:02.563642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-f1d76c29-25c4-45e3-bde7-2ffe719e0e7c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:24:02.563642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:02.563642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:02.563642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:02.563642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:02.563642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:02.563642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:02.563642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:02.563642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:02.563642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:02.563642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:02.563642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:02.563642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:02.563642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:02.563642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:02.563642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:02.564532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:02.564532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:02.564532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:02.564532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:02.564532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:02.564532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:02.564532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:02.564532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:02.564532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:02.564532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:02.564532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:02.564532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:02.564532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:02.564532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:02.564532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:02.564532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:02.565302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:02.565302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:02.565302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:02.565302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:02.565302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:02.565302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:02.565302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:02.565302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:02.565302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:02.565302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:02.565302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:24:02.565302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:02.565302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 134/261] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:24:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:02.574311 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d76c29-25c4-45e3-bde7-2ffe719e0e7c None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:02.693396 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49c3f3-40c7-40a8-9182-fa749c639ab2 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:02.722125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:02.761661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1df97-60a5-47dc-ac43-a8ea2ec281a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:02.763274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1df97-60a5-47dc-ac43-a8ea2ec281a1 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:02.763856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1df97-60a5-47dc-ac43-a8ea2ec281a1 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:02.765942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d76c29-25c4-45e3-bde7-2ffe719e0e7c None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:02.766429 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1df97-60a5-47dc-ac43-a8ea2ec281a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:02.840880 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3176d-a32d-4f02-bf78-06ee15599644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:02.841515 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3176d-a32d-4f02-bf78-06ee15599644 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:02.841797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3176d-a32d-4f02-bf78-06ee15599644 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:02.842072 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3176d-a32d-4f02-bf78-06ee15599644 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:02.842746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3176d-a32d-4f02-bf78-06ee15599644 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:02.843098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3176d-a32d-4f02-bf78-06ee15599644 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:02.844448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3176d-a32d-4f02-bf78-06ee15599644 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:02.857668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 135/262] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:02.862842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3176d-a32d-4f02-bf78-06ee15599644 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:02.887605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:02.898802 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1df97-60a5-47dc-ac43-a8ea2ec281a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:02.900584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1df97-60a5-47dc-ac43-a8ea2ec281a1 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:03.048284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3176d-a32d-4f02-bf78-06ee15599644 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:03.111251 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 128/263] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 686 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:03.111753 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aac31f-a5fa-4344-9fd6-b5c285801da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:03.112442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aac31f-a5fa-4344-9fd6-b5c285801da6 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:03.112721 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aac31f-a5fa-4344-9fd6-b5c285801da6 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:03.112993 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aac31f-a5fa-4344-9fd6-b5c285801da6 None admin] PATH_INFO: `/v3/users/970d0be8205f4a45a498452f92f1768c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:03.114161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aac31f-a5fa-4344-9fd6-b5c285801da6 None admin] RBAC: Authorizing `identity:get_user(user_id=970d0be8205f4a45a498452f92f1768c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:03.115620 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aac31f-a5fa-4344-9fd6-b5c285801da6 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:03.118200 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1df97-60a5-47dc-ac43-a8ea2ec281a1 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:03.121617 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 136/264] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:24:02 2022] GET /identity/v3/users/970d0be8205f4a45a498452f92f1768c => generated 283 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:03.122910 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aac31f-a5fa-4344-9fd6-b5c285801da6 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:03.140475 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:03.259937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aac31f-a5fa-4344-9fd6-b5c285801da6 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:03.299162 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eaabc6-8b8d-4a22-a043-aa482f3a287e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:03.300172 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eaabc6-8b8d-4a22-a043-aa482f3a287e None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:03.300657 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eaabc6-8b8d-4a22-a043-aa482f3a287e None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:03.301126 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eaabc6-8b8d-4a22-a043-aa482f3a287e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:03.373860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:03.403753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1df97-60a5-47dc-ac43-a8ea2ec281a1 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:03.408437 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eaabc6-8b8d-4a22-a043-aa482f3a287e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:03.409963 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eaabc6-8b8d-4a22-a043-aa482f3a287e None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:03.420587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:03.510446 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee73c4d-d819-4c73-81d1-9e99788d230b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:03.511317 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee73c4d-d819-4c73-81d1-9e99788d230b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:03.511650 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee73c4d-d819-4c73-81d1-9e99788d230b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:03.511934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee73c4d-d819-4c73-81d1-9e99788d230b None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:03.516886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee73c4d-d819-4c73-81d1-9e99788d230b None admin] RBAC: Authorizing `identity:get_project(project_id=7d4cfda3691c4a4ab82dda7546659db6)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:03.518379 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee73c4d-d819-4c73-81d1-9e99788d230b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:03.520365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 129/265] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:03 2022] GET /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6 => generated 302 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:03.520945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 137/266] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 22:24:03.530287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee73c4d-d819-4c73-81d1-9e99788d230b None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:03.532439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eaabc6-8b8d-4a22-a043-aa482f3a287e None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:03.692646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eaabc6-8b8d-4a22-a043-aa482f3a287e None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:03.704902 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:03.740978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee73c4d-d819-4c73-81d1-9e99788d230b None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:03.763428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a1092-6fe0-4759-8d10-32ae3cb911c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:03.764306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a1092-6fe0-4759-8d10-32ae3cb911c7 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:03.764717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a1092-6fe0-4759-8d10-32ae3cb911c7 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:03.765116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a1092-6fe0-4759-8d10-32ae3cb911c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:03.765994 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a1092-6fe0-4759-8d10-32ae3cb911c7 None admin] RBAC: Adding query filter params (role.id=a836108447dc439e994611bc74d53d9c, scope.project.id=7d4cfda3691c4a4ab82dda7546659db6, user.id=970d0be8205f4a45a498452f92f1768c) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:03.767221 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a1092-6fe0-4759-8d10-32ae3cb911c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:03.769929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a1092-6fe0-4759-8d10-32ae3cb911c7 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:03.783377 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 138/267] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:24:03 2022] GET /identity/v3/role_assignments?user.id=970d0be8205f4a45a498452f92f1768c&scope.project.id=7d4cfda3691c4a4ab82dda7546659db6&role.id=a836108447dc439e994611bc74d53d9c => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:03.784854 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd31bf5-2680-40da-b0eb-7aa6837ee8ca None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:03.785516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd31bf5-2680-40da-b0eb-7aa6837ee8ca None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:03.785923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd31bf5-2680-40da-b0eb-7aa6837ee8ca None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:03.791461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 139/268] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:03.797736 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364d52a-7c86-4b15-88fb-801200fee382 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:03.799194 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364d52a-7c86-4b15-88fb-801200fee382 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:03.800427 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364d52a-7c86-4b15-88fb-801200fee382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:03.829587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420871e-b41a-4189-b934-e96ff372e563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:03.830562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420871e-b41a-4189-b934-e96ff372e563 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:03.830981 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420871e-b41a-4189-b934-e96ff372e563 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:03.831398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420871e-b41a-4189-b934-e96ff372e563 None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:03.832829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e420871e-b41a-4189-b934-e96ff372e563 None admin] RBAC: Authorizing `identity:get_project(project_id=7d4cfda3691c4a4ab82dda7546659db6)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:03.834533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e420871e-b41a-4189-b934-e96ff372e563 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:03.836644 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 130/269] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:03 2022] GET /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6 => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:03.839448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-1364d52a-7c86-4b15-88fb-801200fee382 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:03.839448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-1364d52a-7c86-4b15-88fb-801200fee382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:03.843780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420871e-b41a-4189-b934-e96ff372e563 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:03.905501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:04.019054 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420871e-b41a-4189-b934-e96ff372e563 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:04.030481 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 140/270] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:04.061391 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979213c3-2d94-47d6-be98-1f4414055a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:04.062294 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979213c3-2d94-47d6-be98-1f4414055a22 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:04.062737 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979213c3-2d94-47d6-be98-1f4414055a22 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:04.063145 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979213c3-2d94-47d6-be98-1f4414055a22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:04.064007 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979213c3-2d94-47d6-be98-1f4414055a22 None admin] RBAC: Adding query filter params (role.id=733cd39702ab4cbc860e34886dbb165f, scope.project.id=7d4cfda3691c4a4ab82dda7546659db6, user.id=970d0be8205f4a45a498452f92f1768c) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:04.064478 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979213c3-2d94-47d6-be98-1f4414055a22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:04.066101 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979213c3-2d94-47d6-be98-1f4414055a22 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:04.077952 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 131/271] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:24:03 2022] GET /identity/v3/role_assignments?user.id=970d0be8205f4a45a498452f92f1768c&scope.project.id=7d4cfda3691c4a4ab82dda7546659db6&role.id=733cd39702ab4cbc860e34886dbb165f => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:04.350671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364d52a-7c86-4b15-88fb-801200fee382 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:04.374054 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e2900-252d-4db9-8175-ecf3f5a38248 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:04.374703 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e2900-252d-4db9-8175-ecf3f5a38248 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:04.375181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e2900-252d-4db9-8175-ecf3f5a38248 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:04.376787 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 132/272] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:24:04.382228 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2370a3b-574e-4821-bbc5-cf1de09b1b56 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:04.382767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2370a3b-574e-4821-bbc5-cf1de09b1b56 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:04.383260 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2370a3b-574e-4821-bbc5-cf1de09b1b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:04.413046 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-e2370a3b-574e-4821-bbc5-cf1de09b1b56 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:04.414541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e2370a3b-574e-4821-bbc5-cf1de09b1b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:04.473860 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:04.476667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364d52a-7c86-4b15-88fb-801200fee382 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:04.498455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 133/273] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:24:04.500090 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecf409-314b-431b-8e26-ec5e005346c8 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:04.501074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecf409-314b-431b-8e26-ec5e005346c8 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:04.501520 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecf409-314b-431b-8e26-ec5e005346c8 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:04.503405 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 134/274] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:04.509789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252320b-84aa-46bd-9d05-dc245a5bb136 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:04.510469 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252320b-84aa-46bd-9d05-dc245a5bb136 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:04.511418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252320b-84aa-46bd-9d05-dc245a5bb136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:04.519282 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f9b36-b4cb-4f4c-aa6c-008e0e9b36b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:04.520241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f9b36-b4cb-4f4c-aa6c-008e0e9b36b6 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:04.520790 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f9b36-b4cb-4f4c-aa6c-008e0e9b36b6 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:04.521402 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f9b36-b4cb-4f4c-aa6c-008e0e9b36b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:04.532683 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f9b36-b4cb-4f4c-aa6c-008e0e9b36b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:04.533007 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-1252320b-84aa-46bd-9d05-dc245a5bb136 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:04.534305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-1252320b-84aa-46bd-9d05-dc245a5bb136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:04.538235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-328f9b36-b4cb-4f4c-aa6c-008e0e9b36b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:04.538235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:04.538235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:04.538235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:04.538235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:04.538235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:04.538235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:04.538235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:04.538235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:04.538235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:04.538235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:04.538235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:04.538235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:04.538235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:04.538235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:04.538235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:04.539110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:04.539110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:04.539110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:04.539110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:04.539110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:04.539110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:04.539110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:04.539110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:04.539110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:04.539110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:04.539110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:04.539110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:04.539110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:04.539110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:04.539110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:04.539110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:04.539966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:04.539966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:04.539966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:04.539966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:04.539966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:04.539966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:04.539966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:04.539966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:04.539966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:04.539966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:04.539966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:04.539966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:04.543297 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 141/275] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:04.549904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f9b36-b4cb-4f4c-aa6c-008e0e9b36b6 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:04.554063 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:04.627876 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 135/276] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:04.670543 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f9b36-b4cb-4f4c-aa6c-008e0e9b36b6 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:04.707167 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d7cee-95b4-4233-8767-a3b4d1cf9f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:04.708619 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d7cee-95b4-4233-8767-a3b4d1cf9f51 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:04.709118 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d7cee-95b4-4233-8767-a3b4d1cf9f51 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:04.709559 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d7cee-95b4-4233-8767-a3b4d1cf9f51 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:04.710390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d7cee-95b4-4233-8767-a3b4d1cf9f51 None admin] RBAC: Adding query filter params (name=member) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:04.710884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d7cee-95b4-4233-8767-a3b4d1cf9f51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:04.712518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d7cee-95b4-4233-8767-a3b4d1cf9f51 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:04.727346 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 142/277] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:24:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:04.728976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252320b-84aa-46bd-9d05-dc245a5bb136 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:04.786385 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d7cee-95b4-4233-8767-a3b4d1cf9f51 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:04.815948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:04.859503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252320b-84aa-46bd-9d05-dc245a5bb136 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:04.900451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29447d-a448-41b0-9c27-d36aad2cef1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:04.901157 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29447d-a448-41b0-9c27-d36aad2cef1b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:04.902051 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29447d-a448-41b0-9c27-d36aad2cef1b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:04.902380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29447d-a448-41b0-9c27-d36aad2cef1b None admin] PATH_INFO: `/v3/users/dd5a206d22294472ab7e4d4852987c70` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:04.911448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d7cee-95b4-4233-8767-a3b4d1cf9f51 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:04.919289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29447d-a448-41b0-9c27-d36aad2cef1b None admin] RBAC: Authorizing `identity:get_user(user_id=dd5a206d22294472ab7e4d4852987c70)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:04.920835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29447d-a448-41b0-9c27-d36aad2cef1b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:04.927258 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 136/278] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:24:04 2022] GET /identity/v3/users/dd5a206d22294472ab7e4d4852987c70 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:04.927258 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29447d-a448-41b0-9c27-d36aad2cef1b None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:04.943856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:04.956447 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde7c44-fb3f-4cac-84a4-209288d02e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:04.957187 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde7c44-fb3f-4cac-84a4-209288d02e12 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:04.957466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde7c44-fb3f-4cac-84a4-209288d02e12 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:04.957827 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde7c44-fb3f-4cac-84a4-209288d02e12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:04.958872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde7c44-fb3f-4cac-84a4-209288d02e12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:04.960311 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde7c44-fb3f-4cac-84a4-209288d02e12 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:04.962094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 143/279] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:04.966440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde7c44-fb3f-4cac-84a4-209288d02e12 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:05.029728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29447d-a448-41b0-9c27-d36aad2cef1b None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:05.059084 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:05.063924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee2f97-b6bb-4503-a9df-d0a234b68b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:05.064715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee2f97-b6bb-4503-a9df-d0a234b68b42 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:05.075205 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee2f97-b6bb-4503-a9df-d0a234b68b42 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:05.075205 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee2f97-b6bb-4503-a9df-d0a234b68b42 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:05.075417 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde7c44-fb3f-4cac-84a4-209288d02e12 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:05.076169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee2f97-b6bb-4503-a9df-d0a234b68b42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:05.080947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-23ee2f97-b6bb-4503-a9df-d0a234b68b42 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:24:05.080947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:05.080947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:05.080947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:05.080947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:05.080947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:05.080947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:05.080947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:05.080947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:05.080947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:05.080947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:05.080947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:05.080947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:05.080947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:05.080947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:05.080947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:05.081855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:05.081855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:05.081855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:05.081855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:05.081855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:05.081855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:05.081855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:05.081855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:05.081855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:05.081855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:05.081855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:05.081855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:05.081855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:05.081855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:05.081855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:05.081855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:05.082655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:05.082655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:05.082655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:05.082655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:05.082655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:05.082655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:05.082655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:05.082655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:05.082655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:05.082655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:05.082655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:24:05.082655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:05.082655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 137/280] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:05.083357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ff424-38ca-41d4-8741-2e0a5bedb7a6 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:05.083422 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ff424-38ca-41d4-8741-2e0a5bedb7a6 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:05.083674 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ff424-38ca-41d4-8741-2e0a5bedb7a6 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:05.085159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 138/281] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:05.085836 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ff424-38ca-41d4-8741-2e0a5bedb7a6 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:05.109260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85c7d8-a51c-42fe-9e96-ea9a12d5e51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:05.109862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85c7d8-a51c-42fe-9e96-ea9a12d5e51c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:05.110138 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85c7d8-a51c-42fe-9e96-ea9a12d5e51c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:05.110402 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85c7d8-a51c-42fe-9e96-ea9a12d5e51c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:05.111115 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85c7d8-a51c-42fe-9e96-ea9a12d5e51c None admin] RBAC: Adding query filter params (role.id=454bd427dfc04030a4ec2b0e8a1ed237, user.id=dd5a206d22294472ab7e4d4852987c70, scope.system=all) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:05.111419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85c7d8-a51c-42fe-9e96-ea9a12d5e51c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:05.112869 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85c7d8-a51c-42fe-9e96-ea9a12d5e51c None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:05.117845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 144/282] 10.222.0.59 () {56 vars in 1308 bytes} [Mon Aug 29 22:24:04 2022] GET /identity/v3/role_assignments?user.id=dd5a206d22294472ab7e4d4852987c70&scope.system=all&role.id=454bd427dfc04030a4ec2b0e8a1ed237 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:05.132396 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85c7d8-a51c-42fe-9e96-ea9a12d5e51c None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:05.165227 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:05.199813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ff424-38ca-41d4-8741-2e0a5bedb7a6 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:05.241557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85c7d8-a51c-42fe-9e96-ea9a12d5e51c None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:05.241867 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb358ba8-aa61-4df8-a7d3-28d0d883d6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:05.242488 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb358ba8-aa61-4df8-a7d3-28d0d883d6f5 None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:05.242767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb358ba8-aa61-4df8-a7d3-28d0d883d6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:05.243092 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb358ba8-aa61-4df8-a7d3-28d0d883d6f5 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:05.243754 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb358ba8-aa61-4df8-a7d3-28d0d883d6f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:05.246092 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb358ba8-aa61-4df8-a7d3-28d0d883d6f5 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:05.282274 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b92d0c-101c-4d21-b0a9-7984ea2ac941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:05.282547 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b92d0c-101c-4d21-b0a9-7984ea2ac941 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:05.282783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b92d0c-101c-4d21-b0a9-7984ea2ac941 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:05.283018 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b92d0c-101c-4d21-b0a9-7984ea2ac941 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:05.283695 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b92d0c-101c-4d21-b0a9-7984ea2ac941 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:05.283997 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b92d0c-101c-4d21-b0a9-7984ea2ac941 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:05.285425 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b92d0c-101c-4d21-b0a9-7984ea2ac941 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:05.295301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 145/283] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:24:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:05.296706 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0c24d-c019-452e-b03e-afb66e9048fa None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:05.296993 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0c24d-c019-452e-b03e-afb66e9048fa None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:05.297329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0c24d-c019-452e-b03e-afb66e9048fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:05.316943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-b6c0c24d-c019-452e-b03e-afb66e9048fa None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:05.318278 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b6c0c24d-c019-452e-b03e-afb66e9048fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:05.330004 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:05.379688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 139/284] 10.222.0.59 () {60 vars in 1132 bytes} [Mon Aug 29 22:24:05 2022] POST /identity/v3/users => generated 293 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:24:05.387639 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 146/285] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:05.389270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb358ba8-aa61-4df8-a7d3-28d0d883d6f5 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:05.525979 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb358ba8-aa61-4df8-a7d3-28d0d883d6f5 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:05.554855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:05.564693 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8f0d8-9230-4a13-ac76-17c2816f20ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:05.565576 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8f0d8-9230-4a13-ac76-17c2816f20ef None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:05.568003 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8f0d8-9230-4a13-ac76-17c2816f20ef None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:05.568003 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8f0d8-9230-4a13-ac76-17c2816f20ef None admin] PATH_INFO: `/v3/users/2743aeab95ff4ba790b49339ce90d5ce` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:05.568003 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8f0d8-9230-4a13-ac76-17c2816f20ef None admin] RBAC: Authorizing `identity:get_user(user_id=2743aeab95ff4ba790b49339ce90d5ce)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:05.569314 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8f0d8-9230-4a13-ac76-17c2816f20ef None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:05.571129 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 140/286] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:24:05 2022] GET /identity/v3/users/2743aeab95ff4ba790b49339ce90d5ce => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:05.577342 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0c24d-c019-452e-b03e-afb66e9048fa None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:05.688916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0c24d-c019-452e-b03e-afb66e9048fa None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:05.711689 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8f0d8-9230-4a13-ac76-17c2816f20ef None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:05.732286 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462dbf7-fd86-4d19-8ce0-d6b3a476fcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:05.732286 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462dbf7-fd86-4d19-8ce0-d6b3a476fcca None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:05.732286 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462dbf7-fd86-4d19-8ce0-d6b3a476fcca None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:05.732286 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462dbf7-fd86-4d19-8ce0-d6b3a476fcca None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:05.734198 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462dbf7-fd86-4d19-8ce0-d6b3a476fcca None admin] RBAC: Authorizing `identity:get_project(project_id=7d4cfda3691c4a4ab82dda7546659db6)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:05.736085 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462dbf7-fd86-4d19-8ce0-d6b3a476fcca None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:05.737913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 147/287] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:05 2022] GET /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:05.744040 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462dbf7-fd86-4d19-8ce0-d6b3a476fcca None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:05.769683 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:05.850957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8f0d8-9230-4a13-ac76-17c2816f20ef None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:05.866737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462dbf7-fd86-4d19-8ce0-d6b3a476fcca None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:05.890666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31f73e-da54-49b3-8117-fa1a9b2d670c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:05.891293 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31f73e-da54-49b3-8117-fa1a9b2d670c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:05.891571 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31f73e-da54-49b3-8117-fa1a9b2d670c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:05.891839 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31f73e-da54-49b3-8117-fa1a9b2d670c None admin] PATH_INFO: `/v3/roles/member` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:05.904691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31f73e-da54-49b3-8117-fa1a9b2d670c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:05.907028 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867182c-a16e-4829-81fe-961a703cda56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:05.907931 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867182c-a16e-4829-81fe-961a703cda56 None admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:05.908464 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867182c-a16e-4829-81fe-961a703cda56 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:05.908910 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867182c-a16e-4829-81fe-961a703cda56 None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6/users/2743aeab95ff4ba790b49339ce90d5ce/roles/d487417153354d90853ba306bd1edb13` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:05.911017 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867182c-a16e-4829-81fe-961a703cda56 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d4cfda3691c4a4ab82dda7546659db6, user_id=2743aeab95ff4ba790b49339ce90d5ce, role_id=d487417153354d90853ba306bd1edb13)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:05.911993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-0f31f73e-da54-49b3-8117-fa1a9b2d670c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:05.911993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:05.911993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:05.911993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:05.911993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:05.911993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:05.911993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:05.911993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:05.911993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:05.911993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:05.911993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:05.911993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:05.911993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:05.911993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:05.911993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:05.911993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:05.912964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:05.912964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:05.912964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:05.912964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:05.912964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:05.912964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:05.912964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:05.912964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:05.912964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:05.912964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:05.912964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:05.912964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:05.912964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:05.912964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:05.912964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:05.912964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:05.913897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:05.913897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:05.913897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:05.913897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:05.913897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:05.913897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:05.913897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:05.913897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:05.913897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:05.913897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:05.913897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:05.913897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:05.913897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 141/288] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:24:05.915470 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867182c-a16e-4829-81fe-961a703cda56 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:05.925465 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31f73e-da54-49b3-8117-fa1a9b2d670c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:05.990465 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:06.007884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 148/289] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:24:05 2022] PUT /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6/users/2743aeab95ff4ba790b49339ce90d5ce/roles/d487417153354d90853ba306bd1edb13 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:24:06.051539 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31f73e-da54-49b3-8117-fa1a9b2d670c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:06.093805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528a254-52d9-4eac-824e-98baa118842d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:06.094466 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528a254-52d9-4eac-824e-98baa118842d None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:06.094749 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528a254-52d9-4eac-824e-98baa118842d None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:06.095017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528a254-52d9-4eac-824e-98baa118842d None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:06.095683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528a254-52d9-4eac-824e-98baa118842d None admin] RBAC: Adding query filter params (name=member) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:06.095981 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528a254-52d9-4eac-824e-98baa118842d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:06.098002 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528a254-52d9-4eac-824e-98baa118842d None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:06.109189 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 142/290] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:24:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:06.114400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867182c-a16e-4829-81fe-961a703cda56 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:06.234863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:06.267578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867182c-a16e-4829-81fe-961a703cda56 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:06.306213 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb12e90-b007-4e1b-9396-d43668255bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:06.306854 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb12e90-b007-4e1b-9396-d43668255bff None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:06.307131 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb12e90-b007-4e1b-9396-d43668255bff None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:06.307907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb12e90-b007-4e1b-9396-d43668255bff None admin] PATH_INFO: `/v3/users/5489097abadb4c039a0e4283bf7db21e` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:06.309444 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb12e90-b007-4e1b-9396-d43668255bff None admin] RBAC: Authorizing `identity:get_user(user_id=5489097abadb4c039a0e4283bf7db21e)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:06.310893 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb12e90-b007-4e1b-9396-d43668255bff None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:06.313005 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 149/291] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:24:06 2022] GET /identity/v3/users/5489097abadb4c039a0e4283bf7db21e => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:06.319071 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528a254-52d9-4eac-824e-98baa118842d None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:06.441709 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528a254-52d9-4eac-824e-98baa118842d None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:06.468155 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:06.491633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7652f-346d-4ba9-a405-09142aae8044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:06.492277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7652f-346d-4ba9-a405-09142aae8044 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:06.492657 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7652f-346d-4ba9-a405-09142aae8044 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:06.492934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7652f-346d-4ba9-a405-09142aae8044 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:06.586877 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7652f-346d-4ba9-a405-09142aae8044 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:06.588354 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7652f-346d-4ba9-a405-09142aae8044 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:06.724051 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 143/292] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 22:24:06.729712 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb12e90-b007-4e1b-9396-d43668255bff None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:06.844598 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb12e90-b007-4e1b-9396-d43668255bff None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:06.880547 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb2d3d-c519-45ac-ab0e-a59f7728fb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:06.881260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb2d3d-c519-45ac-ab0e-a59f7728fb3d None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:06.881535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb2d3d-c519-45ac-ab0e-a59f7728fb3d None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:06.881799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb2d3d-c519-45ac-ab0e-a59f7728fb3d None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:06.882970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb2d3d-c519-45ac-ab0e-a59f7728fb3d None admin] RBAC: Authorizing `identity:get_project(project_id=7d4cfda3691c4a4ab82dda7546659db6)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:06.884418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb2d3d-c519-45ac-ab0e-a59f7728fb3d None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:06.886881 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 150/293] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:06 2022] GET /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:06.893067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7652f-346d-4ba9-a405-09142aae8044 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:06.933413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:07.048995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7652f-346d-4ba9-a405-09142aae8044 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:07.087058 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347623de-2da1-475d-8f34-2ba9395510f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:07.087662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347623de-2da1-475d-8f34-2ba9395510f3 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:07.087936 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347623de-2da1-475d-8f34-2ba9395510f3 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:07.088236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347623de-2da1-475d-8f34-2ba9395510f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:07.089740 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347623de-2da1-475d-8f34-2ba9395510f3 None admin] RBAC: Adding query filter params (role.id=454bd427dfc04030a4ec2b0e8a1ed237, scope.project.id=7d4cfda3691c4a4ab82dda7546659db6, user.id=5489097abadb4c039a0e4283bf7db21e) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:07.090034 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347623de-2da1-475d-8f34-2ba9395510f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:07.091586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347623de-2da1-475d-8f34-2ba9395510f3 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:07.098864 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 144/294] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:24:06 2022] GET /identity/v3/role_assignments?user.id=5489097abadb4c039a0e4283bf7db21e&scope.project.id=7d4cfda3691c4a4ab82dda7546659db6&role.id=454bd427dfc04030a4ec2b0e8a1ed237 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:07.319276 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aae38f-0d5c-4999-bae3-ef64bc60d77d None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:07.319568 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aae38f-0d5c-4999-bae3-ef64bc60d77d None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:07.319743 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aae38f-0d5c-4999-bae3-ef64bc60d77d None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:07.321109 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 151/295] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:07.326574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac41400-cd80-4614-8d21-6b09087927ba None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:07.326861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac41400-cd80-4614-8d21-6b09087927ba None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:07.327192 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac41400-cd80-4614-8d21-6b09087927ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:07.352047 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-7ac41400-cd80-4614-8d21-6b09087927ba None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:07.353619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-7ac41400-cd80-4614-8d21-6b09087927ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:07.435336 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 145/296] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:24:07.723703 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aae38f-0d5c-4999-bae3-ef64bc60d77d None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:07.738932 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:07.839099 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aae38f-0d5c-4999-bae3-ef64bc60d77d None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:07.875974 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab164ab-03eb-4b08-85c4-86b40ca16e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:07.877372 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab164ab-03eb-4b08-85c4-86b40ca16e18 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:07.877860 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab164ab-03eb-4b08-85c4-86b40ca16e18 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:07.878308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab164ab-03eb-4b08-85c4-86b40ca16e18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:07.890798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab164ab-03eb-4b08-85c4-86b40ca16e18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:07.896128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-9ab164ab-03eb-4b08-85c4-86b40ca16e18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:07.896128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:07.896128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:07.896128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:07.896128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:07.896128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:07.896128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:07.896128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:07.896128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:07.896128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:07.896128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:07.896128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:07.896128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:07.896128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:07.896128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:07.896128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:07.897038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:07.897038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:07.897038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:07.897038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:07.897038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:07.897038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:07.897038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:07.897038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:07.897038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:07.897038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:07.897038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:07.897038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:07.897038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:07.897038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:07.897038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:07.897038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:07.897826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:07.897826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:07.897826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:07.897826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:07.897826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:07.897826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:07.897826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:07.897826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:07.897826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:07.897826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:07.897826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:07.897826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:07.904246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 152/297] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:07.907170 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac41400-cd80-4614-8d21-6b09087927ba None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:07.928535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:08.019145 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac41400-cd80-4614-8d21-6b09087927ba None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:08.057030 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd13586-3546-4d4d-94d0-0189439ea92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:08.058549 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd13586-3546-4d4d-94d0-0189439ea92b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:08.058977 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd13586-3546-4d4d-94d0-0189439ea92b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:08.059408 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd13586-3546-4d4d-94d0-0189439ea92b None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:08.060250 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd13586-3546-4d4d-94d0-0189439ea92b None admin] RBAC: Adding query filter params (name=member) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:08.060697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd13586-3546-4d4d-94d0-0189439ea92b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:08.062228 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd13586-3546-4d4d-94d0-0189439ea92b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:08.069978 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a5e21-1550-4a5a-9bb0-4d47d1116991 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:08.070463 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a5e21-1550-4a5a-9bb0-4d47d1116991 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:08.071067 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a5e21-1550-4a5a-9bb0-4d47d1116991 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:08.072651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 153/298] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:08.076357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 146/299] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:24:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:08.081442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264375f-196a-4fc8-adbc-d3c33fb32470 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:08.081714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264375f-196a-4fc8-adbc-d3c33fb32470 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:08.082542 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264375f-196a-4fc8-adbc-d3c33fb32470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:08.085335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd13586-3546-4d4d-94d0-0189439ea92b None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:08.101228 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-8264375f-196a-4fc8-adbc-d3c33fb32470 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:08.102256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-8264375f-196a-4fc8-adbc-d3c33fb32470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:08.178126 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:08.186095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:08.188215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 154/300] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:08.218914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd13586-3546-4d4d-94d0-0189439ea92b None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:08.246278 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e195235-2a94-44d7-80d9-804b3994f399 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:08.246909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e195235-2a94-44d7-80d9-804b3994f399 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:08.247431 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e195235-2a94-44d7-80d9-804b3994f399 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:08.249380 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 155/301] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:24:08.255428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97317d86-a919-4669-bf88-cda46a962533 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:08.255704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97317d86-a919-4669-bf88-cda46a962533 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:08.255975 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97317d86-a919-4669-bf88-cda46a962533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:08.261520 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f3fe2-3af6-4f14-b83f-7f31a2391149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:08.262407 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f3fe2-3af6-4f14-b83f-7f31a2391149 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:08.263097 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f3fe2-3af6-4f14-b83f-7f31a2391149 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:08.263593 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f3fe2-3af6-4f14-b83f-7f31a2391149 None admin] PATH_INFO: `/v3/users/dd5a206d22294472ab7e4d4852987c70` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:08.265419 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f3fe2-3af6-4f14-b83f-7f31a2391149 None admin] RBAC: Authorizing `identity:get_user(user_id=dd5a206d22294472ab7e4d4852987c70)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:08.266998 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f3fe2-3af6-4f14-b83f-7f31a2391149 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:08.269144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 147/302] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:24:08 2022] GET /identity/v3/users/dd5a206d22294472ab7e4d4852987c70 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:08.275101 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f3fe2-3af6-4f14-b83f-7f31a2391149 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:08.278915 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-97317d86-a919-4669-bf88-cda46a962533 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:08.280771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-97317d86-a919-4669-bf88-cda46a962533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:08.350673 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 156/303] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:08.386562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f3fe2-3af6-4f14-b83f-7f31a2391149 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:08.423833 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fffcf-7b34-4f9d-9aa9-885ce6fd39d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:08.424878 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fffcf-7b34-4f9d-9aa9-885ce6fd39d4 None admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:08.425384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fffcf-7b34-4f9d-9aa9-885ce6fd39d4 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:08.425806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fffcf-7b34-4f9d-9aa9-885ce6fd39d4 None admin] PATH_INFO: `/v3/system/users/dd5a206d22294472ab7e4d4852987c70/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:08.427366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921fffcf-7b34-4f9d-9aa9-885ce6fd39d4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=dd5a206d22294472ab7e4d4852987c70, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:08.429050 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921fffcf-7b34-4f9d-9aa9-885ce6fd39d4 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:08.473106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97317d86-a919-4669-bf88-cda46a962533 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:08.539579 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 148/304] 10.222.0.59 () {58 vars in 1334 bytes} [Mon Aug 29 22:24:08 2022] PUT /identity/v3/system/users/dd5a206d22294472ab7e4d4852987c70/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:24:08.540813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93d70b-3096-4e5b-a333-ab4f40e52d7d None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:08.541096 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93d70b-3096-4e5b-a333-ab4f40e52d7d None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:08.541367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93d70b-3096-4e5b-a333-ab4f40e52d7d None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:08.543110 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 149/305] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:08.549195 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c114a5-347b-4759-9289-1435a8f824f9 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:08.549479 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c114a5-347b-4759-9289-1435a8f824f9 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:08.549753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c114a5-347b-4759-9289-1435a8f824f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:08.565682 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:08.570418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-35c114a5-347b-4759-9289-1435a8f824f9 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:08.571677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-35c114a5-347b-4759-9289-1435a8f824f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:08.591664 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97317d86-a919-4669-bf88-cda46a962533 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:08.626088 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:08.631946 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb3db5-d9ee-41a0-8c7b-0bac7efa708e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:08.632626 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb3db5-d9ee-41a0-8c7b-0bac7efa708e None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:08.632941 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb3db5-d9ee-41a0-8c7b-0bac7efa708e None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:08.633210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb3db5-d9ee-41a0-8c7b-0bac7efa708e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:08.644786 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb3db5-d9ee-41a0-8c7b-0bac7efa708e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:08.649903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-e9bb3db5-d9ee-41a0-8c7b-0bac7efa708e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:24:08.649903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:08.649903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:08.649903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:08.649903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:08.649903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:08.649903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:08.649903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:08.649903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:08.649903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:08.649903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:08.649903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:08.649903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:08.649903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:08.649903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:08.649903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:08.653465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:08.653465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:08.653465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:08.653465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:08.653465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:08.653465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:08.653465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:08.653465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:08.653465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:08.653465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:08.653465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:08.653465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:08.653465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:08.653465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:08.653465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:08.653465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:08.654269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:08.654269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:08.654269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:08.654269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:08.654269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:08.654269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:08.654269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:08.654269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:08.654269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:08.654269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:08.654269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:24:08.654269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:08.656625 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 157/306] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:24:08.657379 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 150/307] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:24:08.658748 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb3db5-d9ee-41a0-8c7b-0bac7efa708e None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:08.661210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c114a5-347b-4759-9289-1435a8f824f9 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:08.786048 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c114a5-347b-4759-9289-1435a8f824f9 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:08.787957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb3db5-d9ee-41a0-8c7b-0bac7efa708e None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:08.827619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1e459-1345-4010-bc93-996a8eea19cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:08.828210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1e459-1345-4010-bc93-996a8eea19cf None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:08.828487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1e459-1345-4010-bc93-996a8eea19cf None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:08.828753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1e459-1345-4010-bc93-996a8eea19cf None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:08.829461 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1e459-1345-4010-bc93-996a8eea19cf None admin] RBAC: Adding query filter params (name=reader) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:08.829753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1e459-1345-4010-bc93-996a8eea19cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:08.831128 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1e459-1345-4010-bc93-996a8eea19cf None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:08.834286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:08.836041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d539c4-b487-45cf-911d-1f7758bdee34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:08.836714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d539c4-b487-45cf-911d-1f7758bdee34 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:08.836991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d539c4-b487-45cf-911d-1f7758bdee34 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:08.837267 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d539c4-b487-45cf-911d-1f7758bdee34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:08.839448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d539c4-b487-45cf-911d-1f7758bdee34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:08.841501 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d539c4-b487-45cf-911d-1f7758bdee34 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:08.847060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 151/308] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:24:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:08.847174 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 158/309] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:08.849043 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1e459-1345-4010-bc93-996a8eea19cf None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:08.850688 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d539c4-b487-45cf-911d-1f7758bdee34 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:08.921082 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:08.970284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d539c4-b487-45cf-911d-1f7758bdee34 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:08.977931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1e459-1345-4010-bc93-996a8eea19cf None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:09.004742 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ef5dc-6b31-4614-8293-30b9cdcbd635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:09.005357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ef5dc-6b31-4614-8293-30b9cdcbd635 None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:09.005638 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ef5dc-6b31-4614-8293-30b9cdcbd635 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:09.005908 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ef5dc-6b31-4614-8293-30b9cdcbd635 None admin] PATH_INFO: `/v3/groups` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:09.006526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ef5dc-6b31-4614-8293-30b9cdcbd635 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:09.007952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ef5dc-6b31-4614-8293-30b9cdcbd635 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:09.017290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3041eac-8f35-4fbe-9fbc-38740251fc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:09.017984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3041eac-8f35-4fbe-9fbc-38740251fc12 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:09.018261 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3041eac-8f35-4fbe-9fbc-38740251fc12 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:09.018526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3041eac-8f35-4fbe-9fbc-38740251fc12 None admin] PATH_INFO: `/v3/users/0a706cce0f1140ecb3dbc955e3b7d79b` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:09.034467 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3041eac-8f35-4fbe-9fbc-38740251fc12 None admin] RBAC: Authorizing `identity:get_user(user_id=0a706cce0f1140ecb3dbc955e3b7d79b)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:09.036098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3041eac-8f35-4fbe-9fbc-38740251fc12 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:09.037960 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 152/310] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:24:08 2022] GET /identity/v3/users/0a706cce0f1140ecb3dbc955e3b7d79b => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:09.038554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3041eac-8f35-4fbe-9fbc-38740251fc12 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:09.087942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 159/311] 10.222.0.59 () {60 vars in 1134 bytes} [Mon Aug 29 22:24:08 2022] POST /identity/v3/groups => generated 229 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:24:09.089325 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ef5dc-6b31-4614-8293-30b9cdcbd635 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:09.115316 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:09.184195 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3041eac-8f35-4fbe-9fbc-38740251fc12 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:09.236177 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42660c5-b2b9-4d0b-a1c9-e303a9e64ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:09.237418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42660c5-b2b9-4d0b-a1c9-e303a9e64ac0 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:09.238277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42660c5-b2b9-4d0b-a1c9-e303a9e64ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:09.239032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42660c5-b2b9-4d0b-a1c9-e303a9e64ac0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:09.240772 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ef5dc-6b31-4614-8293-30b9cdcbd635 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:09.252362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42660c5-b2b9-4d0b-a1c9-e303a9e64ac0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:09.257246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-a42660c5-b2b9-4d0b-a1c9-e303a9e64ac0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:24:09.257246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:09.257246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:09.257246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:09.257246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:09.257246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:09.257246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:09.257246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:09.257246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:09.257246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:09.257246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:09.257246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:09.257246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:09.257246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:09.257246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:09.257246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:09.260646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:09.260646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:09.260646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:09.260646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:09.260646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:09.260646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:09.260646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:09.260646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:09.260646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:09.260646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:09.260646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:09.260646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:09.260646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:09.260646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:09.260646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:09.260646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:09.261583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:09.261583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:09.261583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:09.261583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:09.261583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:09.261583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:09.261583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:09.261583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:09.261583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:09.261583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:09.261583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:24:09.261583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:09.261583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 153/312] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:09.265113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42660c5-b2b9-4d0b-a1c9-e303a9e64ac0 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:09.303533 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:09.323860 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb715f85-c7f8-4062-86c0-563386d83c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:09.324535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb715f85-c7f8-4062-86c0-563386d83c72 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:09.324811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb715f85-c7f8-4062-86c0-563386d83c72 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:09.325079 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb715f85-c7f8-4062-86c0-563386d83c72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:09.325841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb715f85-c7f8-4062-86c0-563386d83c72 None admin] RBAC: Adding query filter params (role.id=d487417153354d90853ba306bd1edb13, user.id=0a706cce0f1140ecb3dbc955e3b7d79b, scope.system=all) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:09.326146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb715f85-c7f8-4062-86c0-563386d83c72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:09.327593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb715f85-c7f8-4062-86c0-563386d83c72 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:09.334704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 160/313] 10.222.0.59 () {56 vars in 1308 bytes} [Mon Aug 29 22:24:09 2022] GET /identity/v3/role_assignments?user.id=0a706cce0f1140ecb3dbc955e3b7d79b&scope.system=all&role.id=d487417153354d90853ba306bd1edb13 => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:09.396071 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42660c5-b2b9-4d0b-a1c9-e303a9e64ac0 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:09.417560 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:09.438712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520daf14-cecf-4766-936e-a0822b5e12fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:09.439740 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520daf14-cecf-4766-936e-a0822b5e12fb None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:09.440255 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520daf14-cecf-4766-936e-a0822b5e12fb None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:09.440781 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520daf14-cecf-4766-936e-a0822b5e12fb None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:09.441896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520daf14-cecf-4766-936e-a0822b5e12fb None admin] RBAC: Adding query filter params (name=reader) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:09.442414 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520daf14-cecf-4766-936e-a0822b5e12fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:09.444016 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520daf14-cecf-4766-936e-a0822b5e12fb None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:09.454146 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 154/314] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:24:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:09.461150 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb715f85-c7f8-4062-86c0-563386d83c72 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:09.644696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb715f85-c7f8-4062-86c0-563386d83c72 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:09.703952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05578a8a-51f3-4473-98cd-8c27ada8a75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:09.704935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05578a8a-51f3-4473-98cd-8c27ada8a75b None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:09.705407 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05578a8a-51f3-4473-98cd-8c27ada8a75b None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:09.705885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05578a8a-51f3-4473-98cd-8c27ada8a75b None admin] PATH_INFO: `/v3/users/2743aeab95ff4ba790b49339ce90d5ce` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:09.707796 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05578a8a-51f3-4473-98cd-8c27ada8a75b None admin] RBAC: Authorizing `identity:get_user(user_id=2743aeab95ff4ba790b49339ce90d5ce)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:09.710593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05578a8a-51f3-4473-98cd-8c27ada8a75b None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:09.714534 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 161/315] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:24:09 2022] GET /identity/v3/users/2743aeab95ff4ba790b49339ce90d5ce => generated 297 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:09.720086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520daf14-cecf-4766-936e-a0822b5e12fb None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:09.831088 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520daf14-cecf-4766-936e-a0822b5e12fb None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:09.874455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef9d83-70f9-411a-b5cd-51027a52be8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:09.874455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef9d83-70f9-411a-b5cd-51027a52be8c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:09.874455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef9d83-70f9-411a-b5cd-51027a52be8c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:09.874455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef9d83-70f9-411a-b5cd-51027a52be8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:09.955861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:09.961016 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef9d83-70f9-411a-b5cd-51027a52be8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:09.962502 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef9d83-70f9-411a-b5cd-51027a52be8c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:10.070909 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 155/316] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:10.078191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05578a8a-51f3-4473-98cd-8c27ada8a75b None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:10.134091 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:10.187073 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05578a8a-51f3-4473-98cd-8c27ada8a75b None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:10.222532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c58f6-2304-44f5-a3dc-8730377c6ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:10.223091 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c58f6-2304-44f5-a3dc-8730377c6ebd None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:10.223369 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c58f6-2304-44f5-a3dc-8730377c6ebd None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:10.223640 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c58f6-2304-44f5-a3dc-8730377c6ebd None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:10.224822 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c58f6-2304-44f5-a3dc-8730377c6ebd None admin] RBAC: Authorizing `identity:get_project(project_id=7d4cfda3691c4a4ab82dda7546659db6)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:10.226233 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c58f6-2304-44f5-a3dc-8730377c6ebd None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:10.228042 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 162/317] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:10 2022] GET /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:10.235239 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef9d83-70f9-411a-b5cd-51027a52be8c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:10.319555 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:10.359124 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef9d83-70f9-411a-b5cd-51027a52be8c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:10.423744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1e215-0faf-41ef-bfcc-699c95614ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:10.424798 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1e215-0faf-41ef-bfcc-699c95614ddf None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:10.425271 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1e215-0faf-41ef-bfcc-699c95614ddf None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:10.425754 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1e215-0faf-41ef-bfcc-699c95614ddf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:10.427106 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1e215-0faf-41ef-bfcc-699c95614ddf None admin] RBAC: Adding query filter params (role.id=d487417153354d90853ba306bd1edb13, scope.project.id=7d4cfda3691c4a4ab82dda7546659db6, user.id=2743aeab95ff4ba790b49339ce90d5ce) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:10.427659 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1e215-0faf-41ef-bfcc-699c95614ddf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:10.430339 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1e215-0faf-41ef-bfcc-699c95614ddf None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:10.442527 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 156/318] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:24:10 2022] GET /identity/v3/role_assignments?user.id=2743aeab95ff4ba790b49339ce90d5ce&scope.project.id=7d4cfda3691c4a4ab82dda7546659db6&role.id=d487417153354d90853ba306bd1edb13 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:11.265269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91136847-0b07-4a59-a0c6-2d018595ddea None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:11.265774 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91136847-0b07-4a59-a0c6-2d018595ddea None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:11.266126 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91136847-0b07-4a59-a0c6-2d018595ddea None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:11.269719 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 163/319] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:11.273990 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d447873-2c1b-4d64-9c3f-014c08426416 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:11.274400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d447873-2c1b-4d64-9c3f-014c08426416 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:11.274584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d447873-2c1b-4d64-9c3f-014c08426416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:11.293562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-6d447873-2c1b-4d64-9c3f-014c08426416 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:11.294752 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6d447873-2c1b-4d64-9c3f-014c08426416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:11.362976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 157/320] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:11.408645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a78fc3-69ef-43db-a338-81833e73402d None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:11.408954 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a78fc3-69ef-43db-a338-81833e73402d None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:11.409190 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a78fc3-69ef-43db-a338-81833e73402d None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:11.410659 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 164/321] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:11.415549 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f2ff1-347c-4d76-a33f-8a2e37850849 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:11.415832 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f2ff1-347c-4d76-a33f-8a2e37850849 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:11.416101 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f2ff1-347c-4d76-a33f-8a2e37850849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:11.435701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-597f2ff1-347c-4d76-a33f-8a2e37850849 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:11.436895 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-597f2ff1-347c-4d76-a33f-8a2e37850849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:11.506392 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 158/322] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:11.636100 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a78fc3-69ef-43db-a338-81833e73402d None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:11.742830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a78fc3-69ef-43db-a338-81833e73402d None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:11.760362 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:11.778030 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5195f-c7f2-42d5-93d6-6df7b42c2076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:11.778660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5195f-c7f2-42d5-93d6-6df7b42c2076 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:11.779192 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5195f-c7f2-42d5-93d6-6df7b42c2076 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:11.779293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5195f-c7f2-42d5-93d6-6df7b42c2076 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:11.783920 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f2ff1-347c-4d76-a33f-8a2e37850849 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:11.783920 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:11.795523 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d5195f-c7f2-42d5-93d6-6df7b42c2076 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:11.800500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-b9d5195f-c7f2-42d5-93d6-6df7b42c2076 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:11.800500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:11.800500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:11.800500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:11.800500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:11.800500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:11.800500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:11.800500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:11.800500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:11.800500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:11.800500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:11.800500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:11.800500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:11.800500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:11.800500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:11.800500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:11.801401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:11.801401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:11.801401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:11.801401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:11.801401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:11.801401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:11.801401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:11.801401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:11.801401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:11.801401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:11.801401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:11.801401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:11.801401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:11.801401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:11.801401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:11.801401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:11.802182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:11.802182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:11.802182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:11.802182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:11.802182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:11.802182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:11.802182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:11.802182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:11.802182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:11.802182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:11.802182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:11.802182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:11.802182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 165/323] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:11.809116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5195f-c7f2-42d5-93d6-6df7b42c2076 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:11.887019 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f2ff1-347c-4d76-a33f-8a2e37850849 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:11.913885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5195f-c7f2-42d5-93d6-6df7b42c2076 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:11.923493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd5630-906a-44ca-9fde-08ab7b6d5e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:11.924123 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd5630-906a-44ca-9fde-08ab7b6d5e1e None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:11.924426 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd5630-906a-44ca-9fde-08ab7b6d5e1e None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:11.924696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd5630-906a-44ca-9fde-08ab7b6d5e1e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:11.934918 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd5630-906a-44ca-9fde-08ab7b6d5e1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:11.939807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-5bdd5630-906a-44ca-9fde-08ab7b6d5e1e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:24:11.939807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:11.939807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:11.939807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:11.939807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:11.939807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:11.939807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:11.939807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:11.939807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:11.939807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:11.939807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:11.939807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:11.939807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:11.939807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:11.939807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:11.939807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:11.940747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:11.940747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:11.940747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:11.940747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:11.940747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:11.940747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:11.940747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:11.940747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:11.940747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:11.940747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:11.940747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:11.940747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:11.940747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:11.940747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:11.940747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:11.940747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:11.941890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:11.941890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:11.941890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:11.941890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:11.941890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:11.941890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:11.941890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:11.941890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:11.941890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:11.941890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:11.941890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:24:11.941890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:11.941890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 159/324] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:24:11.942835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9f337-39b4-4d6b-96ea-43a254c055ec None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:11.942912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9f337-39b4-4d6b-96ea-43a254c055ec None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:11.943156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9f337-39b4-4d6b-96ea-43a254c055ec None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:11.944535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 160/325] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:11.946661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9f337-39b4-4d6b-96ea-43a254c055ec None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:11.956661 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178dc25b-113e-4419-9c09-c59280490ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:11.957270 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178dc25b-113e-4419-9c09-c59280490ce8 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:11.957549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178dc25b-113e-4419-9c09-c59280490ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:11.957819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178dc25b-113e-4419-9c09-c59280490ce8 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:11.958483 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178dc25b-113e-4419-9c09-c59280490ce8 None admin] RBAC: Adding query filter params (name=member) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:11.958783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178dc25b-113e-4419-9c09-c59280490ce8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:11.960182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178dc25b-113e-4419-9c09-c59280490ce8 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:11.969164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 166/326] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:24:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:11.970315 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cced32-9e71-4c13-9ceb-31c2a18f8d38 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:11.970596 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cced32-9e71-4c13-9ceb-31c2a18f8d38 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:11.970868 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cced32-9e71-4c13-9ceb-31c2a18f8d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:11.990021 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-b6cced32-9e71-4c13-9ceb-31c2a18f8d38 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:11.991264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b6cced32-9e71-4c13-9ceb-31c2a18f8d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:12.016095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:12.029206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:12.056045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9f337-39b4-4d6b-96ea-43a254c055ec None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:12.062849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 167/327] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:12.076295 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cced32-9e71-4c13-9ceb-31c2a18f8d38 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:12.093428 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0232b-bb93-4d60-81e1-332bac8d94f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:12.094024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0232b-bb93-4d60-81e1-332bac8d94f6 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:12.094294 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0232b-bb93-4d60-81e1-332bac8d94f6 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:12.094560 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0232b-bb93-4d60-81e1-332bac8d94f6 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:12.095212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0232b-bb93-4d60-81e1-332bac8d94f6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:12.095504 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0232b-bb93-4d60-81e1-332bac8d94f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:12.096891 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0232b-bb93-4d60-81e1-332bac8d94f6 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:12.106644 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 161/328] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:24:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:12.112118 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0232b-bb93-4d60-81e1-332bac8d94f6 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:12.174479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cced32-9e71-4c13-9ceb-31c2a18f8d38 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:12.212863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c84d71-7f01-45a0-a8fb-45a017cd818a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:12.214423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c84d71-7f01-45a0-a8fb-45a017cd818a None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:12.214953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c84d71-7f01-45a0-a8fb-45a017cd818a None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:12.215528 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c84d71-7f01-45a0-a8fb-45a017cd818a None admin] PATH_INFO: `/v3/users/dd5a206d22294472ab7e4d4852987c70` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:12.216935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c84d71-7f01-45a0-a8fb-45a017cd818a None admin] RBAC: Authorizing `identity:get_user(user_id=dd5a206d22294472ab7e4d4852987c70)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:12.218245 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0232b-bb93-4d60-81e1-332bac8d94f6 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:12.218890 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c84d71-7f01-45a0-a8fb-45a017cd818a None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:12.221444 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 168/329] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:24:12 2022] GET /identity/v3/users/dd5a206d22294472ab7e4d4852987c70 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:12.227241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c84d71-7f01-45a0-a8fb-45a017cd818a None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:12.243042 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:12.255066 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a678ac-3d7c-4b85-bbf9-0cf4c4021849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:12.255687 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a678ac-3d7c-4b85-bbf9-0cf4c4021849 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:12.255959 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a678ac-3d7c-4b85-bbf9-0cf4c4021849 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:12.256260 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a678ac-3d7c-4b85-bbf9-0cf4c4021849 None admin] PATH_INFO: `/v3/users/0a706cce0f1140ecb3dbc955e3b7d79b` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:12.257503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a678ac-3d7c-4b85-bbf9-0cf4c4021849 None admin] RBAC: Authorizing `identity:get_user(user_id=0a706cce0f1140ecb3dbc955e3b7d79b)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:12.258941 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a678ac-3d7c-4b85-bbf9-0cf4c4021849 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:12.266323 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 162/330] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:24:12 2022] GET /identity/v3/users/0a706cce0f1140ecb3dbc955e3b7d79b => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:12.267915 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a678ac-3d7c-4b85-bbf9-0cf4c4021849 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:12.277183 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:12.341578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c84d71-7f01-45a0-a8fb-45a017cd818a None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:12.374149 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a678ac-3d7c-4b85-bbf9-0cf4c4021849 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:12.376777 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8311d-d381-4999-a27c-5de974350d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:12.377370 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8311d-d381-4999-a27c-5de974350d13 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:12.377642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8311d-d381-4999-a27c-5de974350d13 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:12.377936 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8311d-d381-4999-a27c-5de974350d13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:12.378642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8311d-d381-4999-a27c-5de974350d13 None admin] RBAC: Adding query filter params (role.id=454bd427dfc04030a4ec2b0e8a1ed237, user.id=dd5a206d22294472ab7e4d4852987c70, scope.system=all) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:12.378939 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8311d-d381-4999-a27c-5de974350d13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:12.380385 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8311d-d381-4999-a27c-5de974350d13 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:12.392406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 169/331] 10.222.0.59 () {56 vars in 1308 bytes} [Mon Aug 29 22:24:12 2022] GET /identity/v3/role_assignments?user.id=dd5a206d22294472ab7e4d4852987c70&scope.system=all&role.id=454bd427dfc04030a4ec2b0e8a1ed237 => generated 510 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:12.393035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8311d-d381-4999-a27c-5de974350d13 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:12.418794 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b77824-87c5-4dbc-afd8-616ab26ff8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:12.419592 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b77824-87c5-4dbc-afd8-616ab26ff8a0 None admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:12.419752 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b77824-87c5-4dbc-afd8-616ab26ff8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:12.420024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b77824-87c5-4dbc-afd8-616ab26ff8a0 None admin] PATH_INFO: `/v3/system/users/0a706cce0f1140ecb3dbc955e3b7d79b/roles/d487417153354d90853ba306bd1edb13` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:12.421574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b77824-87c5-4dbc-afd8-616ab26ff8a0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0a706cce0f1140ecb3dbc955e3b7d79b, role_id=d487417153354d90853ba306bd1edb13)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:12.423000 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b77824-87c5-4dbc-afd8-616ab26ff8a0 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:12.493898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 163/332] 10.222.0.59 () {58 vars in 1334 bytes} [Mon Aug 29 22:24:12 2022] PUT /identity/v3/system/users/0a706cce0f1140ecb3dbc955e3b7d79b/roles/d487417153354d90853ba306bd1edb13 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:24:12.501957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:12.509723 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8311d-d381-4999-a27c-5de974350d13 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:12.553198 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043e5ee-3a4a-487a-9298-dc434d4e2d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:12.553847 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043e5ee-3a4a-487a-9298-dc434d4e2d80 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:12.554116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043e5ee-3a4a-487a-9298-dc434d4e2d80 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:12.554379 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043e5ee-3a4a-487a-9298-dc434d4e2d80 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:12.555821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043e5ee-3a4a-487a-9298-dc434d4e2d80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:12.557951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043e5ee-3a4a-487a-9298-dc434d4e2d80 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:12.559621 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 170/333] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:12.565550 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b77824-87c5-4dbc-afd8-616ab26ff8a0 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:12.633162 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:12.678679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b77824-87c5-4dbc-afd8-616ab26ff8a0 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:12.714509 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b339f-b021-42c9-8228-59f30a0f3f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:12.715256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b339f-b021-42c9-8228-59f30a0f3f57 None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:12.715798 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b339f-b021-42c9-8228-59f30a0f3f57 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:12.716331 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b339f-b021-42c9-8228-59f30a0f3f57 None admin] PATH_INFO: `/v3/groups` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:12.717016 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b339f-b021-42c9-8228-59f30a0f3f57 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:12.718974 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b339f-b021-42c9-8228-59f30a0f3f57 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:12.819574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 164/334] 10.222.0.59 () {60 vars in 1134 bytes} [Mon Aug 29 22:24:12 2022] POST /identity/v3/groups => generated 226 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:24:14.566645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f19a8f-6024-48b3-93b0-9b74676b636b None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:14.567152 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f19a8f-6024-48b3-93b0-9b74676b636b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:14.567227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f19a8f-6024-48b3-93b0-9b74676b636b None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:14.568771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 171/335] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:14.575096 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b3409-532a-43a0-b72d-baafcaee4260 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:14.575572 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b3409-532a-43a0-b72d-baafcaee4260 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:14.575649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b3409-532a-43a0-b72d-baafcaee4260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:14.605942 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-a61b3409-532a-43a0-b72d-baafcaee4260 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:14.607162 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a61b3409-532a-43a0-b72d-baafcaee4260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:14.701578 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 165/336] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:15.097764 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c773270-afce-43a3-846e-d71b75fbf283 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:15.098415 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c773270-afce-43a3-846e-d71b75fbf283 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:15.098894 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c773270-afce-43a3-846e-d71b75fbf283 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:15.100855 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 172/337] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:15.106230 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a56ef7-fd8c-46e1-8eac-42680931c75c None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:15.106825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a56ef7-fd8c-46e1-8eac-42680931c75c None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:15.107305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a56ef7-fd8c-46e1-8eac-42680931c75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:15.129489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-93a56ef7-fd8c-46e1-8eac-42680931c75c None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:15.131041 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-93a56ef7-fd8c-46e1-8eac-42680931c75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:15.187406 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:15.216004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 166/338] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:24:15.257014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c773270-afce-43a3-846e-d71b75fbf283 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:15.297332 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8e626-944b-437c-8e44-e30f312e6fa4 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:15.297871 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8e626-944b-437c-8e44-e30f312e6fa4 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:15.298302 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8e626-944b-437c-8e44-e30f312e6fa4 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:15.300219 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 167/339] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:24:15.305404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698870c-240d-4ab0-878a-1e02cca2b195 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:15.305886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698870c-240d-4ab0-878a-1e02cca2b195 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:15.306311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698870c-240d-4ab0-878a-1e02cca2b195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:15.342680 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-d698870c-240d-4ab0-878a-1e02cca2b195 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:15.344910 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-d698870c-240d-4ab0-878a-1e02cca2b195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:15.491682 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 168/340] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:24:15.521763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c773270-afce-43a3-846e-d71b75fbf283 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:15.569792 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dcdd1-8478-4b95-a309-144878e61a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:15.570423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dcdd1-8478-4b95-a309-144878e61a8a None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:15.570685 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dcdd1-8478-4b95-a309-144878e61a8a None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:15.570949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dcdd1-8478-4b95-a309-144878e61a8a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:15.592390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0dcdd1-8478-4b95-a309-144878e61a8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:15.598023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-eb0dcdd1-8478-4b95-a309-144878e61a8a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:24:15.598023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:15.598023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:15.598023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:15.598023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:15.598023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:15.598023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:15.598023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:15.598023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:15.598023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:15.598023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:15.598023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:15.598023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:15.598023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:15.598023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:15.598023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:15.598948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:15.598948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:15.598948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:15.598948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:15.598948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:15.598948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:15.598948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:15.598948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:15.598948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:15.598948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:15.598948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:15.598948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:15.598948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:15.598948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:15.598948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:15.598948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:15.600227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:15.600227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:15.600227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:15.600227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:15.600227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:15.600227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:15.600227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:15.600227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:15.600227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:15.600227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:15.600227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 22:24:15.600227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:15.600227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 173/341] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:15.607953 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698870c-240d-4ab0-878a-1e02cca2b195 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:15.637367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dcdd1-8478-4b95-a309-144878e61a8a None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:15.701121 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:15.817597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698870c-240d-4ab0-878a-1e02cca2b195 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:15.828312 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:15.866027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dcdd1-8478-4b95-a309-144878e61a8a None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:15.885607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d706b-cea1-440d-86d7-92fb88f1b381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:15.886359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d706b-cea1-440d-86d7-92fb88f1b381 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:15.886634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d706b-cea1-440d-86d7-92fb88f1b381 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:15.886896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d706b-cea1-440d-86d7-92fb88f1b381 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:15.887558 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d706b-cea1-440d-86d7-92fb88f1b381 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:15.887853 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d706b-cea1-440d-86d7-92fb88f1b381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:15.889239 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d706b-cea1-440d-86d7-92fb88f1b381 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:15.911752 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 169/342] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:24:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:15.920458 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d706b-cea1-440d-86d7-92fb88f1b381 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:15.942240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce69b65-a16e-4f87-a0b4-079197e832d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:15.942874 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce69b65-a16e-4f87-a0b4-079197e832d4 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:15.943145 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce69b65-a16e-4f87-a0b4-079197e832d4 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:15.943409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce69b65-a16e-4f87-a0b4-079197e832d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:15.973190 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce69b65-a16e-4f87-a0b4-079197e832d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:15.978807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-2ce69b65-a16e-4f87-a0b4-079197e832d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:15.978807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:15.978807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:15.978807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:15.978807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:15.978807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:15.978807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:15.978807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:15.978807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:15.978807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:15.978807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:15.978807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:15.978807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:15.978807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:15.978807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:15.978807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:15.979862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:15.979862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:15.979862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:15.979862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:15.979862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:15.979862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:15.979862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:15.979862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:15.979862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:15.979862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:15.979862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:15.979862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:15.979862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:15.979862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:15.979862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:15.979862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:15.981650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:15.981650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:15.981650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:15.981650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:15.981650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:15.981650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:15.981650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:15.981650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:15.981650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:15.981650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:15.981650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:15.981650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:15.981650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 174/343] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:15.981650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce69b65-a16e-4f87-a0b4-079197e832d4 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:16.120269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce69b65-a16e-4f87-a0b4-079197e832d4 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:16.124504 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:16.128609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d706b-cea1-440d-86d7-92fb88f1b381 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:16.161490 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6110a-15d8-49a4-a9a8-40c6344d7e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:16.162124 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6110a-15d8-49a4-a9a8-40c6344d7e78 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:16.162399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6110a-15d8-49a4-a9a8-40c6344d7e78 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:16.163590 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6110a-15d8-49a4-a9a8-40c6344d7e78 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:16.190048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6110a-15d8-49a4-a9a8-40c6344d7e78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:16.196917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-a6e6110a-15d8-49a4-a9a8-40c6344d7e78 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:24:16.196917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:16.196917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:16.196917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:16.196917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:16.196917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:16.196917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:16.196917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:16.196917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:16.196917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:16.196917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:16.196917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:16.196917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:16.196917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:16.196917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:16.196917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:16.198479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:16.198479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:16.198479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:16.198479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:16.198479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:16.198479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:16.198479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:16.198479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:16.198479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:16.198479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:16.198479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:16.198479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:16.198479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:16.198479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:16.198479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:16.198479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:16.199388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:16.199388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:16.199388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:16.199388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:16.199388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:16.199388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:16.199388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:16.199388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:16.199388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:16.199388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:16.199388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:24:16.199388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:16.204164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 175/344] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:24:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:16.204164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6110a-15d8-49a4-a9a8-40c6344d7e78 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:16.204514 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1d048-bb64-43c4-ad9b-34a4ce73fc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:16.205091 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1d048-bb64-43c4-ad9b-34a4ce73fc9f None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:16.205363 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1d048-bb64-43c4-ad9b-34a4ce73fc9f None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:16.205630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1d048-bb64-43c4-ad9b-34a4ce73fc9f None admin] PATH_INFO: `/v3/users/0a706cce0f1140ecb3dbc955e3b7d79b` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:16.207119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad1d048-bb64-43c4-ad9b-34a4ce73fc9f None admin] RBAC: Authorizing `identity:get_user(user_id=0a706cce0f1140ecb3dbc955e3b7d79b)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:16.208590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad1d048-bb64-43c4-ad9b-34a4ce73fc9f None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:16.213896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 170/345] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:24:15 2022] GET /identity/v3/users/0a706cce0f1140ecb3dbc955e3b7d79b => generated 293 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:16.214888 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1d048-bb64-43c4-ad9b-34a4ce73fc9f None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:16.278761 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:16.348740 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1d048-bb64-43c4-ad9b-34a4ce73fc9f None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:16.356654 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6110a-15d8-49a4-a9a8-40c6344d7e78 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:16.388923 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2128d11-204a-467b-9ede-4b6d97e25973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:16.389468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2128d11-204a-467b-9ede-4b6d97e25973 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:16.389742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2128d11-204a-467b-9ede-4b6d97e25973 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:16.390111 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2128d11-204a-467b-9ede-4b6d97e25973 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:16.390879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2128d11-204a-467b-9ede-4b6d97e25973 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:16.391179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2128d11-204a-467b-9ede-4b6d97e25973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:16.392636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2128d11-204a-467b-9ede-4b6d97e25973 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:16.396019 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2a41a-9b3c-4698-b0e5-425eb500f33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:16.396735 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2a41a-9b3c-4698-b0e5-425eb500f33a None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:16.397010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2a41a-9b3c-4698-b0e5-425eb500f33a None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:16.397280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2a41a-9b3c-4698-b0e5-425eb500f33a None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:16.397959 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2a41a-9b3c-4698-b0e5-425eb500f33a None admin] RBAC: Adding query filter params (name=member) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:16.398258 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2a41a-9b3c-4698-b0e5-425eb500f33a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:16.400723 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2a41a-9b3c-4698-b0e5-425eb500f33a None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:16.403445 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 171/346] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:16.405527 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2128d11-204a-467b-9ede-4b6d97e25973 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:16.411852 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 176/347] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:24:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:16.417631 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150b722-2b2f-4b33-a64a-d5f90e4fbb71 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:16.418034 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150b722-2b2f-4b33-a64a-d5f90e4fbb71 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:16.418316 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150b722-2b2f-4b33-a64a-d5f90e4fbb71 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:16.419691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 177/348] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:16.420497 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150b722-2b2f-4b33-a64a-d5f90e4fbb71 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:16.434117 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:16.554704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150b722-2b2f-4b33-a64a-d5f90e4fbb71 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:16.565997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:16.571496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2128d11-204a-467b-9ede-4b6d97e25973 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:16.599718 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60dcfc6-2fa4-4f85-baa4-31b83de32768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:16.600471 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dcfc6-2fa4-4f85-baa4-31b83de32768 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:16.600697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dcfc6-2fa4-4f85-baa4-31b83de32768 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:16.604236 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dcfc6-2fa4-4f85-baa4-31b83de32768 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:16.626019 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40ed09-367f-43d8-977b-b93c856a58b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:16.626725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40ed09-367f-43d8-977b-b93c856a58b8 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:16.626955 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40ed09-367f-43d8-977b-b93c856a58b8 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:16.627229 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40ed09-367f-43d8-977b-b93c856a58b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:16.627975 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40ed09-367f-43d8-977b-b93c856a58b8 None admin] RBAC: Adding query filter params (role.id=d487417153354d90853ba306bd1edb13, user.id=0a706cce0f1140ecb3dbc955e3b7d79b, scope.system=all) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:16.628358 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40ed09-367f-43d8-977b-b93c856a58b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:16.630804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40ed09-367f-43d8-977b-b93c856a58b8 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:16.647794 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 172/349] 10.222.0.59 () {56 vars in 1308 bytes} [Mon Aug 29 22:24:16 2022] GET /identity/v3/role_assignments?user.id=0a706cce0f1140ecb3dbc955e3b7d79b&scope.system=all&role.id=d487417153354d90853ba306bd1edb13 => generated 510 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:16.649170 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f9f15-a26d-4b86-8d99-f51228b4a51d None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:16.649448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f9f15-a26d-4b86-8d99-f51228b4a51d None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:16.649715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f9f15-a26d-4b86-8d99-f51228b4a51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:16.669707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-512f9f15-a26d-4b86-8d99-f51228b4a51d None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:16.670992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-512f9f15-a26d-4b86-8d99-f51228b4a51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:16.693781 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60dcfc6-2fa4-4f85-baa4-31b83de32768 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:16.695791 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60dcfc6-2fa4-4f85-baa4-31b83de32768 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:16.706792 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:16.793353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 173/350] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:16.794557 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f9f15-a26d-4b86-8d99-f51228b4a51d None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:16.868017 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 178/351] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:16.869171 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d323d21-d69d-4194-a96b-bc00321efd31 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:16.869482 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d323d21-d69d-4194-a96b-bc00321efd31 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:16.870082 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d323d21-d69d-4194-a96b-bc00321efd31 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:16.872665 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 179/352] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:16.879246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0079f0-c6fd-47d1-8398-37f4c4db73aa None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:16.879501 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0079f0-c6fd-47d1-8398-37f4c4db73aa None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:16.879777 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0079f0-c6fd-47d1-8398-37f4c4db73aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:16.903599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-7c0079f0-c6fd-47d1-8398-37f4c4db73aa None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:16.904755 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-7c0079f0-c6fd-47d1-8398-37f4c4db73aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:16.958933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:16.976887 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f9f15-a26d-4b86-8d99-f51228b4a51d None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:16.996674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 180/353] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:17.000174 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0079f0-c6fd-47d1-8398-37f4c4db73aa None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:17.039476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:17.045937 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7735e-7afb-4a4b-8420-687a4adf4820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:17.046908 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7735e-7afb-4a4b-8420-687a4adf4820 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:17.047387 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7735e-7afb-4a4b-8420-687a4adf4820 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:17.047816 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7735e-7afb-4a4b-8420-687a4adf4820 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:17.118195 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0079f0-c6fd-47d1-8398-37f4c4db73aa None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:17.164467 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ee338-000c-459f-9b8c-3ef5b72a432d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:17.165049 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ee338-000c-459f-9b8c-3ef5b72a432d None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:17.165327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ee338-000c-459f-9b8c-3ef5b72a432d None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:17.165599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ee338-000c-459f-9b8c-3ef5b72a432d None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:17.167985 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ee338-000c-459f-9b8c-3ef5b72a432d None admin] RBAC: Authorizing `identity:get_project(project_id=09c1bb7fd3204dffae9576cd2681a4a3)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:17.170043 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ee338-000c-459f-9b8c-3ef5b72a432d None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:17.172330 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 181/354] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:16 2022] GET /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:17.185502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ee338-000c-459f-9b8c-3ef5b72a432d None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:17.270993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f7735e-7afb-4a4b-8420-687a4adf4820 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:17.272493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f7735e-7afb-4a4b-8420-687a4adf4820 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:17.333549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ee338-000c-459f-9b8c-3ef5b72a432d None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:17.336831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:17.376718 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbee3c-689c-4f8b-b098-5568604fcdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:17.377948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbee3c-689c-4f8b-b098-5568604fcdb4 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:17.378431 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbee3c-689c-4f8b-b098-5568604fcdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:17.378921 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbee3c-689c-4f8b-b098-5568604fcdb4 None admin] PATH_INFO: `/v3/groups/9989165196cc40a19882bb7d70ac92e3` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:17.386316 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbee3c-689c-4f8b-b098-5568604fcdb4 None admin] RBAC: Authorizing `identity:get_group(group_id=9989165196cc40a19882bb7d70ac92e3)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:17.388217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbee3c-689c-4f8b-b098-5568604fcdb4 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:17.390597 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 182/355] 10.222.0.59 () {56 vars in 1180 bytes} [Mon Aug 29 22:24:17 2022] GET /identity/v3/groups/9989165196cc40a19882bb7d70ac92e3 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:17.391599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbee3c-689c-4f8b-b098-5568604fcdb4 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:17.467445 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:17.496830 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 174/356] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 703 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:17.498332 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2755b1-6dbf-4a09-855f-2e18311ac9ff None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:17.498601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2755b1-6dbf-4a09-855f-2e18311ac9ff None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:17.499762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2755b1-6dbf-4a09-855f-2e18311ac9ff None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:17.501049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 175/357] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:17.501742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2755b1-6dbf-4a09-855f-2e18311ac9ff None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:17.573331 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbee3c-689c-4f8b-b098-5568604fcdb4 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:17.629688 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84651a70-bf3a-4525-9513-d970c37a909b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:17.630321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84651a70-bf3a-4525-9513-d970c37a909b None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:17.630549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84651a70-bf3a-4525-9513-d970c37a909b None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:17.630795 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84651a70-bf3a-4525-9513-d970c37a909b None admin] PATH_INFO: `/v3/roles/member` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:17.652526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84651a70-bf3a-4525-9513-d970c37a909b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:17.661982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-84651a70-bf3a-4525-9513-d970c37a909b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:17.661982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:17.661982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:17.661982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:17.661982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:17.661982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:17.661982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:17.661982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:17.661982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:17.661982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:17.661982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:17.661982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:17.661982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:17.661982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:17.661982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:17.661982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:17.662889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:17.662889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:17.662889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:17.662889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:17.662889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:17.662889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:17.662889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:17.662889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:17.662889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:17.662889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:17.662889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:17.662889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:17.662889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:17.662889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:17.662889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:17.662889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:17.663689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:17.663689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:17.663689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:17.663689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:17.663689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:17.663689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:17.663689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:17.663689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:17.663689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:17.663689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:17.663689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:17.663689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:17.664467 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 183/358] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:17.665399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84651a70-bf3a-4525-9513-d970c37a909b None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:17.691144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2755b1-6dbf-4a09-855f-2e18311ac9ff None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:17.703961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:17.738440 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fd466-5223-4177-9bd1-14669e36bf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:17.739275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fd466-5223-4177-9bd1-14669e36bf88 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:17.739547 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fd466-5223-4177-9bd1-14669e36bf88 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:17.739812 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fd466-5223-4177-9bd1-14669e36bf88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:17.740632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fd466-5223-4177-9bd1-14669e36bf88 None admin] RBAC: Adding query filter params (group.id=9989165196cc40a19882bb7d70ac92e3, role.id=733cd39702ab4cbc860e34886dbb165f, scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:17.740931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fd466-5223-4177-9bd1-14669e36bf88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:17.742624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fd466-5223-4177-9bd1-14669e36bf88 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:17.750257 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 176/359] 10.222.0.59 () {56 vars in 1376 bytes} [Mon Aug 29 22:24:17 2022] GET /identity/v3/role_assignments?group.id=9989165196cc40a19882bb7d70ac92e3&scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3&role.id=733cd39702ab4cbc860e34886dbb165f => generated 260 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:17.751854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a8c9f-6355-4175-8989-ca5e811a4c79 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:17.752159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a8c9f-6355-4175-8989-ca5e811a4c79 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:17.752491 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a8c9f-6355-4175-8989-ca5e811a4c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:17.778695 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-005a8c9f-6355-4175-8989-ca5e811a4c79 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:17.780372 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-005a8c9f-6355-4175-8989-ca5e811a4c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:17.806116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84651a70-bf3a-4525-9513-d970c37a909b None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:17.845569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:17.860424 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f3047-5c1b-40de-98b8-d18c60873f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:17.861348 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f3047-5c1b-40de-98b8-d18c60873f0d None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:17.861794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f3047-5c1b-40de-98b8-d18c60873f0d None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:17.863069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f3047-5c1b-40de-98b8-d18c60873f0d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:17.880123 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f3047-5c1b-40de-98b8-d18c60873f0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:17.885702 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 177/360] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:17.885702 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a8c9f-6355-4175-8989-ca5e811a4c79 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:17.888821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-852f3047-5c1b-40de-98b8-d18c60873f0d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:24:17.888821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:17.888821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:17.888821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:17.888821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:17.888821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:17.888821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:17.888821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:17.888821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:17.888821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:17.888821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:17.888821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:17.888821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:17.888821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:17.888821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:17.888821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:17.889859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:17.889859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:17.889859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:17.889859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:17.889859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:17.889859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:17.889859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:17.889859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:17.889859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:17.889859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:17.889859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:17.889859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:17.889859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:17.889859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:17.889859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:17.889859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:17.890649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:17.890649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:17.890649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:17.890649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:17.890649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:17.890649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:17.890649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:17.890649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:17.890649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:17.890649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:17.890649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:24:17.890649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:17.892981 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 184/361] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:24:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:17.894173 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f3047-5c1b-40de-98b8-d18c60873f0d None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:18.011963 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a8c9f-6355-4175-8989-ca5e811a4c79 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:18.013276 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:18.026701 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f3047-5c1b-40de-98b8-d18c60873f0d None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:18.058275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3139bfd-31ca-4167-9e93-3d6139c5a23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:18.058754 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3139bfd-31ca-4167-9e93-3d6139c5a23a None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:18.059136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3139bfd-31ca-4167-9e93-3d6139c5a23a None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:18.059572 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3139bfd-31ca-4167-9e93-3d6139c5a23a None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:18.061866 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3139bfd-31ca-4167-9e93-3d6139c5a23a None admin] RBAC: Authorizing `identity:get_project(project_id=09c1bb7fd3204dffae9576cd2681a4a3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:18.063460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3139bfd-31ca-4167-9e93-3d6139c5a23a None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:18.065672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 178/362] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:17 2022] GET /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:18.066780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3139bfd-31ca-4167-9e93-3d6139c5a23a None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:18.070272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41410f9-7e7f-492e-aaa9-df0a6f61915e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:18.071734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41410f9-7e7f-492e-aaa9-df0a6f61915e None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:18.071734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41410f9-7e7f-492e-aaa9-df0a6f61915e None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:18.071734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41410f9-7e7f-492e-aaa9-df0a6f61915e None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:18.072541 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41410f9-7e7f-492e-aaa9-df0a6f61915e None admin] RBAC: Adding query filter params (name=member) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:18.072938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41410f9-7e7f-492e-aaa9-df0a6f61915e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:18.074444 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41410f9-7e7f-492e-aaa9-df0a6f61915e None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:18.087709 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 185/363] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:24:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:18.088959 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41410f9-7e7f-492e-aaa9-df0a6f61915e None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:18.126175 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:18.184289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3139bfd-31ca-4167-9e93-3d6139c5a23a None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:18.203668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41410f9-7e7f-492e-aaa9-df0a6f61915e None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:18.223799 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a37493-bd27-4764-85e9-6713676690c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:18.224435 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a37493-bd27-4764-85e9-6713676690c2 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:18.224711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a37493-bd27-4764-85e9-6713676690c2 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:18.224978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a37493-bd27-4764-85e9-6713676690c2 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:18.225638 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a37493-bd27-4764-85e9-6713676690c2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:18.225931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a37493-bd27-4764-85e9-6713676690c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:18.227763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a37493-bd27-4764-85e9-6713676690c2 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:18.239565 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 179/364] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:18.240787 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a37493-bd27-4764-85e9-6713676690c2 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:18.246008 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91cd196-c9f8-4b64-b113-6ef122d641d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:18.246908 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cd196-c9f8-4b64-b113-6ef122d641d5 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:18.247332 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cd196-c9f8-4b64-b113-6ef122d641d5 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:18.247825 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91cd196-c9f8-4b64-b113-6ef122d641d5 None admin] PATH_INFO: `/v3/groups/9989165196cc40a19882bb7d70ac92e3` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:18.249218 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91cd196-c9f8-4b64-b113-6ef122d641d5 None admin] RBAC: Authorizing `identity:get_group(group_id=9989165196cc40a19882bb7d70ac92e3)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:18.251867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91cd196-c9f8-4b64-b113-6ef122d641d5 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:18.253725 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 186/365] 10.222.0.59 () {56 vars in 1180 bytes} [Mon Aug 29 22:24:18 2022] GET /identity/v3/groups/9989165196cc40a19882bb7d70ac92e3 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:18.254309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91cd196-c9f8-4b64-b113-6ef122d641d5 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:18.278389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:18.382401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91cd196-c9f8-4b64-b113-6ef122d641d5 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:18.388583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a37493-bd27-4764-85e9-6713676690c2 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:18.404439 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:18.426411 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59cf55-90e8-4fe9-8750-6fb8523faadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:18.427075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59cf55-90e8-4fe9-8750-6fb8523faadc None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:18.427300 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59cf55-90e8-4fe9-8750-6fb8523faadc None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:18.429546 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59cf55-90e8-4fe9-8750-6fb8523faadc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:18.448040 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3c0a8-5323-47ce-96c4-bbb5e41d4584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:18.449199 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3c0a8-5323-47ce-96c4-bbb5e41d4584 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:18.449199 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3c0a8-5323-47ce-96c4-bbb5e41d4584 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:18.449393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3c0a8-5323-47ce-96c4-bbb5e41d4584 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:18.515609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db59cf55-90e8-4fe9-8750-6fb8523faadc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:18.517818 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db59cf55-90e8-4fe9-8750-6fb8523faadc None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:18.542809 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:18.545315 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3c0a8-5323-47ce-96c4-bbb5e41d4584 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:18.547013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3c0a8-5323-47ce-96c4-bbb5e41d4584 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:18.654003 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 187/366] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:18.654952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59cf55-90e8-4fe9-8750-6fb8523faadc None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:18.678308 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 180/367] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:18.679158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3c0a8-5323-47ce-96c4-bbb5e41d4584 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:18.693738 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:18.784125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59cf55-90e8-4fe9-8750-6fb8523faadc None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:18.805892 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:18.811125 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3c0a8-5323-47ce-96c4-bbb5e41d4584 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:18.828084 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ec9df-034c-414a-9dfe-23fd2f759738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:18.828777 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ec9df-034c-414a-9dfe-23fd2f759738 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:18.829057 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ec9df-034c-414a-9dfe-23fd2f759738 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:18.829326 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ec9df-034c-414a-9dfe-23fd2f759738 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:18.842392 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ec9df-034c-414a-9dfe-23fd2f759738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:18.847472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-a37ec9df-034c-414a-9dfe-23fd2f759738 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:24:18.847472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:18.847472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:18.847472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:18.847472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:18.847472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:18.847472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:18.847472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:18.847472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:18.847472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:18.847472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:18.847472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:18.847472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:18.847472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:18.847472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:18.847472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:18.848416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:18.848416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:18.848416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:18.848416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:18.848416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:18.848416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:18.848416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:18.848416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:18.848416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:18.848416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:18.848416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:18.848416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:18.848416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:18.848416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:18.848416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:18.848416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:18.849189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:18.849189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:18.849189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:18.849189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:18.849189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:18.849189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:18.849189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:18.849189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:18.849189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:18.849189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:18.849189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:24:18.849189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:18.849189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 188/368] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:24:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:18.849849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ec9df-034c-414a-9dfe-23fd2f759738 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:18.861819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed90b1-f6e3-4267-b4d6-e343b23347e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:18.862828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed90b1-f6e3-4267-b4d6-e343b23347e8 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:18.863637 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed90b1-f6e3-4267-b4d6-e343b23347e8 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:18.864095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed90b1-f6e3-4267-b4d6-e343b23347e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:18.865358 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed90b1-f6e3-4267-b4d6-e343b23347e8 None admin] RBAC: Adding query filter params (group.id=9989165196cc40a19882bb7d70ac92e3, role.id=454bd427dfc04030a4ec2b0e8a1ed237, scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:18.869063 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed90b1-f6e3-4267-b4d6-e343b23347e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:18.869063 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed90b1-f6e3-4267-b4d6-e343b23347e8 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:18.876420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 181/369] 10.222.0.59 () {56 vars in 1376 bytes} [Mon Aug 29 22:24:18 2022] GET /identity/v3/role_assignments?group.id=9989165196cc40a19882bb7d70ac92e3&scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3&role.id=454bd427dfc04030a4ec2b0e8a1ed237 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:18.877254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed90b1-f6e3-4267-b4d6-e343b23347e8 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:18.988108 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:19.027565 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ec9df-034c-414a-9dfe-23fd2f759738 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:19.029648 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed90b1-f6e3-4267-b4d6-e343b23347e8 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:19.076320 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42476b-ee3d-43cc-9ba0-f519e3c736b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:19.076889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42476b-ee3d-43cc-9ba0-f519e3c736b6 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:19.077155 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42476b-ee3d-43cc-9ba0-f519e3c736b6 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:19.077423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42476b-ee3d-43cc-9ba0-f519e3c736b6 None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:19.079057 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8bffd-69ea-47df-ae7e-135c6a64882f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:19.079536 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a42476b-ee3d-43cc-9ba0-f519e3c736b6 None admin] RBAC: Authorizing `identity:get_project(project_id=7d4cfda3691c4a4ab82dda7546659db6)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:19.080251 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8bffd-69ea-47df-ae7e-135c6a64882f None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:19.080779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8bffd-69ea-47df-ae7e-135c6a64882f None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:19.081001 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a42476b-ee3d-43cc-9ba0-f519e3c736b6 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:19.081445 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8bffd-69ea-47df-ae7e-135c6a64882f None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:19.084599 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8bffd-69ea-47df-ae7e-135c6a64882f None admin] RBAC: Authorizing `identity:get_project(project_id=7d4cfda3691c4a4ab82dda7546659db6)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:19.084723 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 189/370] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:18 2022] GET /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:19.084723 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42476b-ee3d-43cc-9ba0-f519e3c736b6 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:19.085949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8bffd-69ea-47df-ae7e-135c6a64882f None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:19.088637 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 182/371] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:18 2022] GET /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:19.092954 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8bffd-69ea-47df-ae7e-135c6a64882f None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:19.144107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:19.212843 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42476b-ee3d-43cc-9ba0-f519e3c736b6 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:19.218959 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8bffd-69ea-47df-ae7e-135c6a64882f None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:19.253856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56efed6-2fc6-4745-b4e7-c5247e30f4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:19.254850 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56efed6-2fc6-4745-b4e7-c5247e30f4af None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:19.255274 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56efed6-2fc6-4745-b4e7-c5247e30f4af None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:19.255744 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56efed6-2fc6-4745-b4e7-c5247e30f4af None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:19.256636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56efed6-2fc6-4745-b4e7-c5247e30f4af None admin] RBAC: Adding query filter params (name=admin) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:19.257190 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56efed6-2fc6-4745-b4e7-c5247e30f4af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:19.258754 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56efed6-2fc6-4745-b4e7-c5247e30f4af None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:19.259720 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb28de0-c3a1-4ccf-a10d-440ade7647ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:19.260925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb28de0-c3a1-4ccf-a10d-440ade7647ec None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:19.261357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb28de0-c3a1-4ccf-a10d-440ade7647ec None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:19.261884 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb28de0-c3a1-4ccf-a10d-440ade7647ec None admin] PATH_INFO: `/v3/groups/9989165196cc40a19882bb7d70ac92e3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:19.263289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb28de0-c3a1-4ccf-a10d-440ade7647ec None admin] RBAC: Authorizing `identity:get_group(group_id=9989165196cc40a19882bb7d70ac92e3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:19.265084 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb28de0-c3a1-4ccf-a10d-440ade7647ec None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:19.267324 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 183/372] 10.222.0.59 () {56 vars in 1180 bytes} [Mon Aug 29 22:24:19 2022] GET /identity/v3/groups/9989165196cc40a19882bb7d70ac92e3 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:19.268337 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb28de0-c3a1-4ccf-a10d-440ade7647ec None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:19.269134 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 190/373] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:19.272804 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56efed6-2fc6-4745-b4e7-c5247e30f4af None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:19.285780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:19.397438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56efed6-2fc6-4745-b4e7-c5247e30f4af None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:19.402214 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb28de0-c3a1-4ccf-a10d-440ade7647ec None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:19.408775 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:19.438122 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5a00d-8410-4646-9218-a4713126a8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:19.439081 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5a00d-8410-4646-9218-a4713126a8cf None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:19.439688 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5a00d-8410-4646-9218-a4713126a8cf None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:19.440368 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5a00d-8410-4646-9218-a4713126a8cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:19.441439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5a00d-8410-4646-9218-a4713126a8cf None admin] RBAC: Adding query filter params (group.id=9989165196cc40a19882bb7d70ac92e3, role.id=733cd39702ab4cbc860e34886dbb165f, scope.project.id=7d4cfda3691c4a4ab82dda7546659db6) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:19.441555 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fa488-0164-44cf-a9e0-0b94ba4130b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:19.442074 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5a00d-8410-4646-9218-a4713126a8cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:19.442191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fa488-0164-44cf-a9e0-0b94ba4130b0 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:19.442474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fa488-0164-44cf-a9e0-0b94ba4130b0 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:19.442747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fa488-0164-44cf-a9e0-0b94ba4130b0 None admin] PATH_INFO: `/v3/groups/9989165196cc40a19882bb7d70ac92e3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:19.443886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1fa488-0164-44cf-a9e0-0b94ba4130b0 None admin] RBAC: Authorizing `identity:get_group(group_id=9989165196cc40a19882bb7d70ac92e3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:19.444488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5a00d-8410-4646-9218-a4713126a8cf None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:19.445702 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1fa488-0164-44cf-a9e0-0b94ba4130b0 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:19.452378 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 184/374] 10.222.0.59 () {56 vars in 1180 bytes} [Mon Aug 29 22:24:19 2022] GET /identity/v3/groups/9989165196cc40a19882bb7d70ac92e3 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:19.453972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fa488-0164-44cf-a9e0-0b94ba4130b0 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:19.458444 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 191/375] 10.222.0.59 () {56 vars in 1376 bytes} [Mon Aug 29 22:24:19 2022] GET /identity/v3/role_assignments?group.id=9989165196cc40a19882bb7d70ac92e3&scope.project.id=7d4cfda3691c4a4ab82dda7546659db6&role.id=733cd39702ab4cbc860e34886dbb165f => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:19.459699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5a00d-8410-4646-9218-a4713126a8cf None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:19.580513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5a00d-8410-4646-9218-a4713126a8cf None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:19.618567 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:19.631683 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fc754-530f-47a0-90e5-1232b129ec06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:19.632685 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fc754-530f-47a0-90e5-1232b129ec06 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:19.633129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fc754-530f-47a0-90e5-1232b129ec06 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:19.635813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fc754-530f-47a0-90e5-1232b129ec06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:19.635813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fc754-530f-47a0-90e5-1232b129ec06 None admin] RBAC: Adding query filter params (group.id=9989165196cc40a19882bb7d70ac92e3, role.id=454bd427dfc04030a4ec2b0e8a1ed237, scope.project.id=7d4cfda3691c4a4ab82dda7546659db6) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:19.635813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fc754-530f-47a0-90e5-1232b129ec06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:19.637162 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fc754-530f-47a0-90e5-1232b129ec06 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:19.648344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 192/376] 10.222.0.59 () {56 vars in 1376 bytes} [Mon Aug 29 22:24:19 2022] GET /identity/v3/role_assignments?group.id=9989165196cc40a19882bb7d70ac92e3&scope.project.id=7d4cfda3691c4a4ab82dda7546659db6&role.id=454bd427dfc04030a4ec2b0e8a1ed237 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:19.649057 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fa488-0164-44cf-a9e0-0b94ba4130b0 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:19.691729 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c97ee-ebf4-4929-a74e-c3038f7c965d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:19.692456 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c97ee-ebf4-4929-a74e-c3038f7c965d None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:19.692737 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c97ee-ebf4-4929-a74e-c3038f7c965d None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:19.693004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c97ee-ebf4-4929-a74e-c3038f7c965d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:19.774776 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c97ee-ebf4-4929-a74e-c3038f7c965d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:19.776692 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c97ee-ebf4-4929-a74e-c3038f7c965d None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:19.881922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:19.911616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 185/377] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:19.917920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fc754-530f-47a0-90e5-1232b129ec06 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:19.981367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:20.046960 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7837caa-6865-41e3-929f-a9a54586cdb5 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:20.047293 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7837caa-6865-41e3-929f-a9a54586cdb5 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:20.047509 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7837caa-6865-41e3-929f-a9a54586cdb5 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:20.048898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 186/378] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:20.055317 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac7989-5ef6-4169-9153-5d117f088ec4 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:20.055840 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac7989-5ef6-4169-9153-5d117f088ec4 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:20.056285 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac7989-5ef6-4169-9153-5d117f088ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:20.091170 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-88ac7989-5ef6-4169-9153-5d117f088ec4 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:20.093977 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-88ac7989-5ef6-4169-9153-5d117f088ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:20.104460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fc754-530f-47a0-90e5-1232b129ec06 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:20.174991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dccda-b431-4833-a8b2-81cd2ae3ef41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:20.175982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dccda-b431-4833-a8b2-81cd2ae3ef41 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:20.176778 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dccda-b431-4833-a8b2-81cd2ae3ef41 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:20.177248 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dccda-b431-4833-a8b2-81cd2ae3ef41 None admin] PATH_INFO: `/v3/projects/d3d78b58a69d4b10868c64742da904fe` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:20.180921 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01dccda-b431-4833-a8b2-81cd2ae3ef41 None admin] RBAC: Authorizing `identity:get_project(project_id=d3d78b58a69d4b10868c64742da904fe)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:20.184011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01dccda-b431-4833-a8b2-81cd2ae3ef41 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:20.186098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 193/379] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:19 2022] GET /identity/v3/projects/d3d78b58a69d4b10868c64742da904fe => generated 344 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:20.197173 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dccda-b431-4833-a8b2-81cd2ae3ef41 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:20.248083 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 187/380] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:24:20.360079 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dccda-b431-4833-a8b2-81cd2ae3ef41 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:20.362711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:20.405215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6937a-c87e-4096-90d5-3986f133e2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:20.405884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6937a-c87e-4096-90d5-3986f133e2d3 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:20.406177 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6937a-c87e-4096-90d5-3986f133e2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:20.406717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6937a-c87e-4096-90d5-3986f133e2d3 None admin] PATH_INFO: `/v3/groups/58bc1551bf5a4d2e9bf9e41461a32198` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:20.413051 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6937a-c87e-4096-90d5-3986f133e2d3 None admin] RBAC: Authorizing `identity:get_group(group_id=58bc1551bf5a4d2e9bf9e41461a32198)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:20.414585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6937a-c87e-4096-90d5-3986f133e2d3 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:20.416409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 194/381] 10.222.0.59 () {56 vars in 1180 bytes} [Mon Aug 29 22:24:20 2022] GET /identity/v3/groups/58bc1551bf5a4d2e9bf9e41461a32198 => generated 229 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:20.422994 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac7989-5ef6-4169-9153-5d117f088ec4 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:20.565033 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac7989-5ef6-4169-9153-5d117f088ec4 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:20.576919 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:20.606768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6b0ff-f0b7-40c2-ad6b-13a531bbc41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:20.607375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6b0ff-f0b7-40c2-ad6b-13a531bbc41c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:20.607647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6b0ff-f0b7-40c2-ad6b-13a531bbc41c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:20.607911 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6b0ff-f0b7-40c2-ad6b-13a531bbc41c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:20.608668 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6b0ff-f0b7-40c2-ad6b-13a531bbc41c None admin] RBAC: Adding query filter params (group.id=58bc1551bf5a4d2e9bf9e41461a32198, role.id=a836108447dc439e994611bc74d53d9c, scope.project.id=d3d78b58a69d4b10868c64742da904fe) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:20.608968 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6b0ff-f0b7-40c2-ad6b-13a531bbc41c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:20.610454 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6b0ff-f0b7-40c2-ad6b-13a531bbc41c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:20.617403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 188/382] 10.222.0.59 () {56 vars in 1376 bytes} [Mon Aug 29 22:24:20 2022] GET /identity/v3/role_assignments?group.id=58bc1551bf5a4d2e9bf9e41461a32198&scope.project.id=d3d78b58a69d4b10868c64742da904fe&role.id=a836108447dc439e994611bc74d53d9c => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:20.635576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6937a-c87e-4096-90d5-3986f133e2d3 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:20.799392 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6937a-c87e-4096-90d5-3986f133e2d3 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:20.847010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aee221-b70f-4f4f-a642-10a140e96f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:20.847575 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aee221-b70f-4f4f-a642-10a140e96f8e None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:20.847848 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aee221-b70f-4f4f-a642-10a140e96f8e None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:20.848114 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aee221-b70f-4f4f-a642-10a140e96f8e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:20.868552 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aee221-b70f-4f4f-a642-10a140e96f8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:20.874895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-25aee221-b70f-4f4f-a642-10a140e96f8e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:24:20.874895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:20.874895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:20.874895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:20.874895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:20.874895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:20.874895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:20.874895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:20.874895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:20.874895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:20.874895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:20.874895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:20.874895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:20.874895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:20.874895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:20.874895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:20.875762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:20.875762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:20.875762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:20.875762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:20.875762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:20.875762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:20.875762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:20.875762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:20.875762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:20.875762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:20.875762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:20.875762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:20.875762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:20.875762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:20.875762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:20.875762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:20.876585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:20.876585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:20.876585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:20.876585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:20.876585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:20.876585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:20.876585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:20.876585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:20.876585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:20.876585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:20.876585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:24:20.876585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:20.876585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 195/383] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:24:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:20.881817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6b0ff-f0b7-40c2-ad6b-13a531bbc41c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:21.021326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6b0ff-f0b7-40c2-ad6b-13a531bbc41c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:21.068265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e9fa0-8ec5-4191-9528-baadde4fc843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:21.069269 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e9fa0-8ec5-4191-9528-baadde4fc843 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:21.069744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e9fa0-8ec5-4191-9528-baadde4fc843 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:21.070232 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e9fa0-8ec5-4191-9528-baadde4fc843 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:21.071086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e9fa0-8ec5-4191-9528-baadde4fc843 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:21.071577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e9fa0-8ec5-4191-9528-baadde4fc843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:21.073729 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e9fa0-8ec5-4191-9528-baadde4fc843 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:21.087669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 189/384] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:21.094319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aee221-b70f-4f4f-a642-10a140e96f8e None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:21.143069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:21.237158 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aee221-b70f-4f4f-a642-10a140e96f8e None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:21.281965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958afc92-cdf4-4b2e-bf4b-0a5a6fe15af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:21.282599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958afc92-cdf4-4b2e-bf4b-0a5a6fe15af3 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:21.282869 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958afc92-cdf4-4b2e-bf4b-0a5a6fe15af3 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:21.283134 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958afc92-cdf4-4b2e-bf4b-0a5a6fe15af3 None admin] PATH_INFO: `/v3/groups/9989165196cc40a19882bb7d70ac92e3` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:21.284264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958afc92-cdf4-4b2e-bf4b-0a5a6fe15af3 None admin] RBAC: Authorizing `identity:get_group(group_id=9989165196cc40a19882bb7d70ac92e3)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:21.285696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958afc92-cdf4-4b2e-bf4b-0a5a6fe15af3 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:21.287671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 196/385] 10.222.0.59 () {56 vars in 1180 bytes} [Mon Aug 29 22:24:21 2022] GET /identity/v3/groups/9989165196cc40a19882bb7d70ac92e3 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:21.293792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e9fa0-8ec5-4191-9528-baadde4fc843 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:21.357711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:21.438925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e9fa0-8ec5-4191-9528-baadde4fc843 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:21.477710 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452f13f-f462-4eee-8c69-3a448c2f2f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:21.479209 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452f13f-f462-4eee-8c69-3a448c2f2f21 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:21.479634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452f13f-f462-4eee-8c69-3a448c2f2f21 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:21.480220 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452f13f-f462-4eee-8c69-3a448c2f2f21 None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:21.481555 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d452f13f-f462-4eee-8c69-3a448c2f2f21 None admin] RBAC: Authorizing `identity:get_project(project_id=09c1bb7fd3204dffae9576cd2681a4a3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:21.483142 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d452f13f-f462-4eee-8c69-3a448c2f2f21 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:21.485382 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 190/386] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:21 2022] GET /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:21.491362 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958afc92-cdf4-4b2e-bf4b-0a5a6fe15af3 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:21.606859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:21.613649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958afc92-cdf4-4b2e-bf4b-0a5a6fe15af3 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:21.622568 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211efff-dc60-46e3-bbd6-8dc29efda1b5 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:21.623192 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211efff-dc60-46e3-bbd6-8dc29efda1b5 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:21.623720 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211efff-dc60-46e3-bbd6-8dc29efda1b5 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:21.626595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 191/387] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:21.631074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d1e66-9674-44a6-b87b-612541f4aaf6 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:21.631600 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d1e66-9674-44a6-b87b-612541f4aaf6 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:21.632079 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d1e66-9674-44a6-b87b-612541f4aaf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:21.654126 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-448d1e66-9674-44a6-b87b-612541f4aaf6 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:21.655780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-448d1e66-9674-44a6-b87b-612541f4aaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:21.665625 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdcb426-a02e-4045-8a51-110b0ed6ab19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:21.667654 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcb426-a02e-4045-8a51-110b0ed6ab19 None admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:21.668158 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcb426-a02e-4045-8a51-110b0ed6ab19 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:21.668858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcb426-a02e-4045-8a51-110b0ed6ab19 None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3/groups/9989165196cc40a19882bb7d70ac92e3/roles/733cd39702ab4cbc860e34886dbb165f` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:21.673349 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdcb426-a02e-4045-8a51-110b0ed6ab19 None admin] RBAC: Authorizing `identity:create_grant(project_id=09c1bb7fd3204dffae9576cd2681a4a3, group_id=9989165196cc40a19882bb7d70ac92e3, role_id=733cd39702ab4cbc860e34886dbb165f)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:21.675232 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdcb426-a02e-4045-8a51-110b0ed6ab19 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:21.746883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 192/388] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:21.760227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 197/389] 10.222.0.59 () {58 vars in 1442 bytes} [Mon Aug 29 22:24:21 2022] PUT /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3/groups/9989165196cc40a19882bb7d70ac92e3/roles/733cd39702ab4cbc860e34886dbb165f => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:24:22.131903 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d1e66-9674-44a6-b87b-612541f4aaf6 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:22.171312 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:22.246821 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d1e66-9674-44a6-b87b-612541f4aaf6 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:22.278188 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf534d-d13c-454b-a27e-054727ea8b79 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:22.279032 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf534d-d13c-454b-a27e-054727ea8b79 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:22.279445 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf534d-d13c-454b-a27e-054727ea8b79 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:22.281293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 198/390] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:22.284224 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf2cd1-24e4-4eb7-ab0a-44d35eabe27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:22.284948 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf2cd1-24e4-4eb7-ab0a-44d35eabe27d None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:22.284948 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf2cd1-24e4-4eb7-ab0a-44d35eabe27d None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:22.284948 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf2cd1-24e4-4eb7-ab0a-44d35eabe27d None admin] PATH_INFO: `/v3/roles/member` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:22.286492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce911f4c-24eb-4a77-ab53-418b582fa12b None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:22.286927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce911f4c-24eb-4a77-ab53-418b582fa12b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:22.287337 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce911f4c-24eb-4a77-ab53-418b582fa12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:22.297349 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf2cd1-24e4-4eb7-ab0a-44d35eabe27d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:22.303015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-dfbf2cd1-24e4-4eb7-ab0a-44d35eabe27d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:22.303015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:22.303015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:22.303015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:22.303015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:22.303015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:22.303015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:22.303015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:22.303015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:22.303015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:22.303015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:22.303015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:22.303015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:22.303015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:22.303015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:22.303015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:22.304102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:22.304102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:22.304102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:22.304102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:22.304102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:22.304102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:22.304102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:22.304102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:22.304102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:22.304102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:22.304102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:22.304102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:22.304102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:22.304102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:22.304102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:22.304102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:22.305098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:22.305098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:22.305098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:22.305098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:22.305098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:22.305098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:22.305098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:22.305098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:22.305098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:22.305098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:22.305098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:22.305098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:22.305098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 193/391] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:22.313403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf2cd1-24e4-4eb7-ab0a-44d35eabe27d None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:22.329198 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-ce911f4c-24eb-4a77-ab53-418b582fa12b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:22.330647 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ce911f4c-24eb-4a77-ab53-418b582fa12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:22.408932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:22.417992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf2cd1-24e4-4eb7-ab0a-44d35eabe27d None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:22.454579 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a523a1c-1b76-4f92-a969-95307051af50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:22.455432 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a523a1c-1b76-4f92-a969-95307051af50 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:22.456056 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a523a1c-1b76-4f92-a969-95307051af50 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:22.456535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a523a1c-1b76-4f92-a969-95307051af50 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:22.457493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a523a1c-1b76-4f92-a969-95307051af50 None admin] RBAC: Adding query filter params (name=member) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:22.457937 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a523a1c-1b76-4f92-a969-95307051af50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:22.459808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a523a1c-1b76-4f92-a969-95307051af50 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:22.476810 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 199/392] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:22.478126 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a758992-155e-455d-8ce6-58c90ea839ee None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:22.478917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a758992-155e-455d-8ce6-58c90ea839ee None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:22.479198 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a758992-155e-455d-8ce6-58c90ea839ee None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:22.479856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 194/393] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:24:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:22.486881 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a523a1c-1b76-4f92-a969-95307051af50 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:22.489558 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 200/394] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:22 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:22.512914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3bf0d-1b49-42b2-9374-7ad24aa75318 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:22.513728 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3bf0d-1b49-42b2-9374-7ad24aa75318 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:22.514417 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3bf0d-1b49-42b2-9374-7ad24aa75318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:22.569370 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-12b3bf0d-1b49-42b2-9374-7ad24aa75318 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:22.570854 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-12b3bf0d-1b49-42b2-9374-7ad24aa75318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:22.626289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a523a1c-1b76-4f92-a969-95307051af50 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:22.663508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8576a34-c685-476c-b707-2e2b95622539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:22.664514 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8576a34-c685-476c-b707-2e2b95622539 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:22.665116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8576a34-c685-476c-b707-2e2b95622539 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:22.665677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8576a34-c685-476c-b707-2e2b95622539 None admin] PATH_INFO: `/v3/groups/9989165196cc40a19882bb7d70ac92e3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:22.667858 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8576a34-c685-476c-b707-2e2b95622539 None admin] RBAC: Authorizing `identity:get_group(group_id=9989165196cc40a19882bb7d70ac92e3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:22.669874 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8576a34-c685-476c-b707-2e2b95622539 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:22.672159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 195/395] 10.222.0.59 () {56 vars in 1180 bytes} [Mon Aug 29 22:24:22 2022] GET /identity/v3/groups/9989165196cc40a19882bb7d70ac92e3 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:22.678575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8576a34-c685-476c-b707-2e2b95622539 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:22.731774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:22.740699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 201/396] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:24:22.820059 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3bf0d-1b49-42b2-9374-7ad24aa75318 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:22.847696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:22.867780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8576a34-c685-476c-b707-2e2b95622539 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:22.942823 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165b9d6-ceff-496d-af4d-8c7305306a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:22.943528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165b9d6-ceff-496d-af4d-8c7305306a80 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:22.943863 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165b9d6-ceff-496d-af4d-8c7305306a80 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:22.944129 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165b9d6-ceff-496d-af4d-8c7305306a80 None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:22.946559 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165b9d6-ceff-496d-af4d-8c7305306a80 None admin] RBAC: Authorizing `identity:get_project(project_id=09c1bb7fd3204dffae9576cd2681a4a3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:22.949578 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165b9d6-ceff-496d-af4d-8c7305306a80 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:22.955672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 196/397] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:22 2022] GET /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3 => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:22.961217 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165b9d6-ceff-496d-af4d-8c7305306a80 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:23.006597 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3bf0d-1b49-42b2-9374-7ad24aa75318 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:23.067160 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780d3b7-1b88-4b22-875b-c2a576531523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:23.067788 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780d3b7-1b88-4b22-875b-c2a576531523 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:23.068066 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780d3b7-1b88-4b22-875b-c2a576531523 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:23.068368 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780d3b7-1b88-4b22-875b-c2a576531523 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:23.080337 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9780d3b7-1b88-4b22-875b-c2a576531523 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:23.085225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-9780d3b7-1b88-4b22-875b-c2a576531523 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:23.085225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:23.085225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:23.085225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:23.085225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:23.085225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:23.085225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:23.085225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:23.085225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:23.085225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:23.085225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:23.085225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:23.085225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:23.085225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:23.085225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:23.085225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:23.086107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:23.086107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:23.086107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:23.086107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:23.086107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:23.086107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:23.086107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:23.086107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:23.086107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:23.086107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:23.086107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:23.086107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:23.086107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:23.086107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:23.086107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:23.086107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:23.086907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:23.086907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:23.086907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:23.086907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:23.086907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:23.086907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:23.086907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:23.086907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:23.086907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:23.086907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:23.086907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:23.086907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:23.086907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 202/398] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:24:23.088421 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e57c19-b7b2-4f51-9fed-92cbf6ce0aec None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:23.088704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e57c19-b7b2-4f51-9fed-92cbf6ce0aec None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:23.088979 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e57c19-b7b2-4f51-9fed-92cbf6ce0aec None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:23.094254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 203/399] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:23.094254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e57c19-b7b2-4f51-9fed-92cbf6ce0aec None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:23.125245 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:23.135417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165b9d6-ceff-496d-af4d-8c7305306a80 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:23.175904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baecf46-4d8e-41b2-a52b-29078118c281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:23.176671 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baecf46-4d8e-41b2-a52b-29078118c281 None admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:23.176952 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baecf46-4d8e-41b2-a52b-29078118c281 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:23.177228 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baecf46-4d8e-41b2-a52b-29078118c281 None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3/groups/9989165196cc40a19882bb7d70ac92e3/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:23.179261 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baecf46-4d8e-41b2-a52b-29078118c281 None admin] RBAC: Authorizing `identity:create_grant(project_id=09c1bb7fd3204dffae9576cd2681a4a3, group_id=9989165196cc40a19882bb7d70ac92e3, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:23.180880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baecf46-4d8e-41b2-a52b-29078118c281 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:23.187937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:23.214013 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e57c19-b7b2-4f51-9fed-92cbf6ce0aec None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:23.252396 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1502f-a6d5-494d-9a11-4a836733e7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:23.252999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1502f-a6d5-494d-9a11-4a836733e7d6 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:23.253272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1502f-a6d5-494d-9a11-4a836733e7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:23.253538 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1502f-a6d5-494d-9a11-4a836733e7d6 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:23.254190 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1502f-a6d5-494d-9a11-4a836733e7d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:23.254483 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1502f-a6d5-494d-9a11-4a836733e7d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:23.255832 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1502f-a6d5-494d-9a11-4a836733e7d6 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:23.264412 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 204/400] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:24:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:23.265552 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad6e3d-8950-4494-9810-13ac620dcec5 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:23.268317 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad6e3d-8950-4494-9810-13ac620dcec5 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:23.268317 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad6e3d-8950-4494-9810-13ac620dcec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:23.292879 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-d3ad6e3d-8950-4494-9810-13ac620dcec5 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:23.294049 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d3ad6e3d-8950-4494-9810-13ac620dcec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:23.328846 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 197/401] 10.222.0.59 () {58 vars in 1442 bytes} [Mon Aug 29 22:24:22 2022] PUT /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3/groups/9989165196cc40a19882bb7d70ac92e3/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:24:23.329439 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baecf46-4d8e-41b2-a52b-29078118c281 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:23.405584 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 205/402] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:23.406668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad6e3d-8950-4494-9810-13ac620dcec5 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:23.486642 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baecf46-4d8e-41b2-a52b-29078118c281 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:23.504527 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:23.526913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ddd6d4-3982-4a85-a882-4839ba8c9eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:23.527552 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddd6d4-3982-4a85-a882-4839ba8c9eb4 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:23.527827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddd6d4-3982-4a85-a882-4839ba8c9eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:23.528093 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddd6d4-3982-4a85-a882-4839ba8c9eb4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:23.539997 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad6e3d-8950-4494-9810-13ac620dcec5 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:23.540195 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ddd6d4-3982-4a85-a882-4839ba8c9eb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:23.545883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-a9ddd6d4-3982-4a85-a882-4839ba8c9eb4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:24:23.545883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:23.545883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:23.545883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:23.545883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:23.545883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:23.545883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:23.545883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:23.545883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:23.545883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:23.545883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:23.545883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:23.545883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:23.545883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:23.545883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:23.545883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:23.546865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:23.546865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:23.546865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:23.546865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:23.546865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:23.546865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:23.546865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:23.546865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:23.546865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:23.546865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:23.546865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:23.546865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:23.546865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:23.546865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:23.546865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:23.546865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:23.547678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:23.547678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:23.547678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:23.547678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:23.547678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:23.547678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:23.547678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:23.547678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:23.547678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:23.547678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:23.547678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:24:23.547678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:23.547678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 198/403] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:24:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:23.553303 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ddd6d4-3982-4a85-a882-4839ba8c9eb4 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:23.571755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:23.584070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766895b-d4ca-4d1e-b561-1e8f65172c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:23.584819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766895b-d4ca-4d1e-b561-1e8f65172c49 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:23.585163 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766895b-d4ca-4d1e-b561-1e8f65172c49 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:23.585521 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766895b-d4ca-4d1e-b561-1e8f65172c49 None admin] PATH_INFO: `/v3/groups/9989165196cc40a19882bb7d70ac92e3` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:23.586854 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766895b-d4ca-4d1e-b561-1e8f65172c49 None admin] RBAC: Authorizing `identity:get_group(group_id=9989165196cc40a19882bb7d70ac92e3)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:23.589283 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766895b-d4ca-4d1e-b561-1e8f65172c49 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:23.591225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 206/404] 10.222.0.59 () {56 vars in 1180 bytes} [Mon Aug 29 22:24:23 2022] GET /identity/v3/groups/9989165196cc40a19882bb7d70ac92e3 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:23.596804 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766895b-d4ca-4d1e-b561-1e8f65172c49 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:23.721608 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ddd6d4-3982-4a85-a882-4839ba8c9eb4 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:23.762749 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766895b-d4ca-4d1e-b561-1e8f65172c49 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:23.764060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c8cd5-31d1-49b2-8ed0-ab8a7d2f8276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:23.764998 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c8cd5-31d1-49b2-8ed0-ab8a7d2f8276 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:23.765355 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c8cd5-31d1-49b2-8ed0-ab8a7d2f8276 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:23.765628 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c8cd5-31d1-49b2-8ed0-ab8a7d2f8276 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:23.766331 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c8cd5-31d1-49b2-8ed0-ab8a7d2f8276 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:23.766877 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c8cd5-31d1-49b2-8ed0-ab8a7d2f8276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:23.768306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c8cd5-31d1-49b2-8ed0-ab8a7d2f8276 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:23.778869 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 199/405] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:23.779897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c8cd5-31d1-49b2-8ed0-ab8a7d2f8276 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:23.805660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cd9cf-4aab-47bc-8e2d-05245ad1ed86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:23.806306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cd9cf-4aab-47bc-8e2d-05245ad1ed86 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:23.806680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cd9cf-4aab-47bc-8e2d-05245ad1ed86 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:23.806952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cd9cf-4aab-47bc-8e2d-05245ad1ed86 None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:23.809056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1cd9cf-4aab-47bc-8e2d-05245ad1ed86 None admin] RBAC: Authorizing `identity:get_project(project_id=7d4cfda3691c4a4ab82dda7546659db6)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:23.811053 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1cd9cf-4aab-47bc-8e2d-05245ad1ed86 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:23.814108 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 207/406] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:23 2022] GET /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:23.814913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cd9cf-4aab-47bc-8e2d-05245ad1ed86 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:23.821355 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:23.874382 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:23.909989 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c8cd5-31d1-49b2-8ed0-ab8a7d2f8276 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:23.953587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef82ab6-172a-4612-ad3b-065adfd92efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:23.954341 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef82ab6-172a-4612-ad3b-065adfd92efc None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:23.954632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef82ab6-172a-4612-ad3b-065adfd92efc None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:23.954926 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef82ab6-172a-4612-ad3b-065adfd92efc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:23.964379 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cd9cf-4aab-47bc-8e2d-05245ad1ed86 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:23.971784 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef82ab6-172a-4612-ad3b-065adfd92efc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:23.978679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-bef82ab6-172a-4612-ad3b-065adfd92efc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:24:23.978679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:23.978679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:23.978679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:23.978679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:23.978679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:23.978679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:23.978679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:23.978679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:23.978679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:23.978679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:23.978679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:23.978679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:23.978679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:23.978679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:23.978679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:23.979746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:23.979746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:23.979746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:23.979746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:23.979746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:23.979746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:23.979746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:23.979746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:23.979746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:23.979746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:23.979746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:23.979746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:23.979746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:23.979746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:23.979746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:23.979746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:23.980587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:23.980587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:23.980587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:23.980587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:23.980587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:23.980587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:23.980587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:23.980587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:23.980587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:23.980587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:23.980587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:24:23.980587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:23.980587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 200/407] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:24:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:23.980587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef82ab6-172a-4612-ad3b-065adfd92efc None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:24.016398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd2573-1b59-4a0f-afb1-b07f7c5ee29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:24.019150 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd2573-1b59-4a0f-afb1-b07f7c5ee29c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:24.019611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd2573-1b59-4a0f-afb1-b07f7c5ee29c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:24.020175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd2573-1b59-4a0f-afb1-b07f7c5ee29c None admin] PATH_INFO: `/v3/groups/9989165196cc40a19882bb7d70ac92e3` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:24.021800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd2573-1b59-4a0f-afb1-b07f7c5ee29c None admin] RBAC: Authorizing `identity:get_group(group_id=9989165196cc40a19882bb7d70ac92e3)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:24.024613 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd2573-1b59-4a0f-afb1-b07f7c5ee29c None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:24.027720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 208/408] 10.222.0.59 () {56 vars in 1180 bytes} [Mon Aug 29 22:24:23 2022] GET /identity/v3/groups/9989165196cc40a19882bb7d70ac92e3 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:24.028842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd2573-1b59-4a0f-afb1-b07f7c5ee29c None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:24.142753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef82ab6-172a-4612-ad3b-065adfd92efc None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:24.147273 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:24.191918 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:24.196938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd2573-1b59-4a0f-afb1-b07f7c5ee29c None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:24.198291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee671d-7619-4c93-974c-c48bfd615856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:24.198967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee671d-7619-4c93-974c-c48bfd615856 None admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:24.199243 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee671d-7619-4c93-974c-c48bfd615856 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:24.199804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee671d-7619-4c93-974c-c48bfd615856 None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6/groups/9989165196cc40a19882bb7d70ac92e3/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:24.203143 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee671d-7619-4c93-974c-c48bfd615856 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d4cfda3691c4a4ab82dda7546659db6, group_id=9989165196cc40a19882bb7d70ac92e3, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:24.204633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee671d-7619-4c93-974c-c48bfd615856 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:24.242886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a89115-9d8d-4ae9-a63d-79bf70396310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:24.243539 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a89115-9d8d-4ae9-a63d-79bf70396310 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:24.243841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a89115-9d8d-4ae9-a63d-79bf70396310 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:24.244255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a89115-9d8d-4ae9-a63d-79bf70396310 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:24.245239 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a89115-9d8d-4ae9-a63d-79bf70396310 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:24.245541 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a89115-9d8d-4ae9-a63d-79bf70396310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:24.247040 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a89115-9d8d-4ae9-a63d-79bf70396310 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:24.257766 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 209/409] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:24.263569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a89115-9d8d-4ae9-a63d-79bf70396310 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:24.311380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 201/410] 10.222.0.59 () {58 vars in 1442 bytes} [Mon Aug 29 22:24:23 2022] PUT /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6/groups/9989165196cc40a19882bb7d70ac92e3/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:24:24.321800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca268c67-6224-4e38-b33c-990efed6a2fe None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:24.322140 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca268c67-6224-4e38-b33c-990efed6a2fe None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:24.322418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca268c67-6224-4e38-b33c-990efed6a2fe None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:24.333949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 202/411] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:24.333949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca268c67-6224-4e38-b33c-990efed6a2fe None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:24.412513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a89115-9d8d-4ae9-a63d-79bf70396310 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:24.464933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe8ebc1-5d95-407e-bf20-8cd0f1fe18f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:24.466151 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8ebc1-5d95-407e-bf20-8cd0f1fe18f4 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:24.466723 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8ebc1-5d95-407e-bf20-8cd0f1fe18f4 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:24.467577 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8ebc1-5d95-407e-bf20-8cd0f1fe18f4 None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:24.469277 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe8ebc1-5d95-407e-bf20-8cd0f1fe18f4 None admin] RBAC: Authorizing `identity:get_project(project_id=7d4cfda3691c4a4ab82dda7546659db6)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:24.472358 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe8ebc1-5d95-407e-bf20-8cd0f1fe18f4 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:24.474220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 210/412] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:24 2022] GET /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:24.475531 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82822b8-0bf9-40d4-b4fb-2216ea088856 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:24.475906 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82822b8-0bf9-40d4-b4fb-2216ea088856 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:24.476310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82822b8-0bf9-40d4-b4fb-2216ea088856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:24.483967 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:24.499932 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-d82822b8-0bf9-40d4-b4fb-2216ea088856 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:24.501477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d82822b8-0bf9-40d4-b4fb-2216ea088856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:24.513962 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca268c67-6224-4e38-b33c-990efed6a2fe None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:24.561158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5210b18-9e3a-44b8-901e-c6eade43c00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:24.562707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5210b18-9e3a-44b8-901e-c6eade43c00c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:24.563482 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5210b18-9e3a-44b8-901e-c6eade43c00c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:24.564257 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5210b18-9e3a-44b8-901e-c6eade43c00c None admin] PATH_INFO: `/v3/groups/58bc1551bf5a4d2e9bf9e41461a32198` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:24.566542 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5210b18-9e3a-44b8-901e-c6eade43c00c None admin] RBAC: Authorizing `identity:get_group(group_id=58bc1551bf5a4d2e9bf9e41461a32198)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:24.569513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5210b18-9e3a-44b8-901e-c6eade43c00c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:24.571624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 203/413] 10.222.0.59 () {56 vars in 1180 bytes} [Mon Aug 29 22:24:24 2022] GET /identity/v3/groups/58bc1551bf5a4d2e9bf9e41461a32198 => generated 229 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:24.577089 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5210b18-9e3a-44b8-901e-c6eade43c00c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:24.605787 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 211/414] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:24.608697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82822b8-0bf9-40d4-b4fb-2216ea088856 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:24.701324 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:24.782604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5210b18-9e3a-44b8-901e-c6eade43c00c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:24.831834 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82822b8-0bf9-40d4-b4fb-2216ea088856 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:24.854206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d6fc8-1b86-4d50-b9be-6a685e0adc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:24.854987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d6fc8-1b86-4d50-b9be-6a685e0adc83 None admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:24.855267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d6fc8-1b86-4d50-b9be-6a685e0adc83 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:24.855536 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d6fc8-1b86-4d50-b9be-6a685e0adc83 None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6/groups/9989165196cc40a19882bb7d70ac92e3/roles/733cd39702ab4cbc860e34886dbb165f` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:24.858954 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d6fc8-1b86-4d50-b9be-6a685e0adc83 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d4cfda3691c4a4ab82dda7546659db6, group_id=9989165196cc40a19882bb7d70ac92e3, role_id=733cd39702ab4cbc860e34886dbb165f)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:24.862122 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d6fc8-1b86-4d50-b9be-6a685e0adc83 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:24.886530 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:24.890211 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a5532-5e83-4138-a1a8-127d5cacd705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:24.891135 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a5532-5e83-4138-a1a8-127d5cacd705 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:24.891635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a5532-5e83-4138-a1a8-127d5cacd705 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:24.892330 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a5532-5e83-4138-a1a8-127d5cacd705 None admin] PATH_INFO: `/v3/projects/d3d78b58a69d4b10868c64742da904fe` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:24.893777 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a5532-5e83-4138-a1a8-127d5cacd705 None admin] RBAC: Authorizing `identity:get_project(project_id=d3d78b58a69d4b10868c64742da904fe)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:24.895684 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a5532-5e83-4138-a1a8-127d5cacd705 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:24.897663 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 212/415] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:24 2022] GET /identity/v3/projects/d3d78b58a69d4b10868c64742da904fe => generated 344 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:24.905050 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a5532-5e83-4138-a1a8-127d5cacd705 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:24.936197 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 204/416] 10.222.0.59 () {58 vars in 1442 bytes} [Mon Aug 29 22:24:24 2022] PUT /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6/groups/9989165196cc40a19882bb7d70ac92e3/roles/733cd39702ab4cbc860e34886dbb165f => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:24:24.966560 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d6fc8-1b86-4d50-b9be-6a685e0adc83 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:25.033797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a5532-5e83-4138-a1a8-127d5cacd705 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:25.075040 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4149d23-9508-4041-bdb5-e46d3f557931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:25.075802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4149d23-9508-4041-bdb5-e46d3f557931 None admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:25.076176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4149d23-9508-4041-bdb5-e46d3f557931 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:25.076442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4149d23-9508-4041-bdb5-e46d3f557931 None admin] PATH_INFO: `/v3/projects/d3d78b58a69d4b10868c64742da904fe/groups/58bc1551bf5a4d2e9bf9e41461a32198/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:25.078312 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d6fc8-1b86-4d50-b9be-6a685e0adc83 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:25.078487 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4149d23-9508-4041-bdb5-e46d3f557931 None admin] RBAC: Authorizing `identity:create_grant(project_id=d3d78b58a69d4b10868c64742da904fe, group_id=58bc1551bf5a4d2e9bf9e41461a32198, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:25.080126 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4149d23-9508-4041-bdb5-e46d3f557931 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:25.106628 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:25.124390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927053c-aab4-42bd-827b-0b193c5d5df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:25.125091 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927053c-aab4-42bd-827b-0b193c5d5df5 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:25.125413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927053c-aab4-42bd-827b-0b193c5d5df5 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:25.125684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927053c-aab4-42bd-827b-0b193c5d5df5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:25.139254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927053c-aab4-42bd-827b-0b193c5d5df5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:25.144647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-6927053c-aab4-42bd-827b-0b193c5d5df5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:24:25.144647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:25.144647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:25.144647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:25.144647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:25.144647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:25.144647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:25.144647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:25.144647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:25.144647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:25.144647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:25.144647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:25.144647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:25.144647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:25.144647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:25.144647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:25.147113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:25.147113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:25.147113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:25.147113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:25.147113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:25.147113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:25.147113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:25.147113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:25.147113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:25.147113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:25.147113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:25.147113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:25.147113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:25.147113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:25.147113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:25.147113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:25.148808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:25.148808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:25.148808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:25.148808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:25.148808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:25.148808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:25.148808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:25.148808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:25.148808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:25.148808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:25.148808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:24:25.148808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:25.148808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 205/417] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:24:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:24:25.155356 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927053c-aab4-42bd-827b-0b193c5d5df5 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:25.172388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 213/418] 10.222.0.59 () {58 vars in 1442 bytes} [Mon Aug 29 22:24:24 2022] PUT /identity/v3/projects/d3d78b58a69d4b10868c64742da904fe/groups/58bc1551bf5a4d2e9bf9e41461a32198/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:24:25.274943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927053c-aab4-42bd-827b-0b193c5d5df5 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:25.316501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb245f1-ebb2-4044-80e9-47f4ad55d953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:25.317129 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb245f1-ebb2-4044-80e9-47f4ad55d953 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:25.317397 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb245f1-ebb2-4044-80e9-47f4ad55d953 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:25.317740 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb245f1-ebb2-4044-80e9-47f4ad55d953 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:25.318488 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb245f1-ebb2-4044-80e9-47f4ad55d953 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:25.318785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb245f1-ebb2-4044-80e9-47f4ad55d953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:25.320267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb245f1-ebb2-4044-80e9-47f4ad55d953 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:25.330182 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 206/419] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:25.335903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4149d23-9508-4041-bdb5-e46d3f557931 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:25.402803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:25.447917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4149d23-9508-4041-bdb5-e46d3f557931 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:25.505053 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef4f1e-4105-4629-bb21-f57e54154693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:25.506027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef4f1e-4105-4629-bb21-f57e54154693 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:25.506503 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef4f1e-4105-4629-bb21-f57e54154693 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:25.506981 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef4f1e-4105-4629-bb21-f57e54154693 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:25.565644 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bedd1-561d-4829-a689-2fa097f4a085 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:25.565964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bedd1-561d-4829-a689-2fa097f4a085 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:25.566192 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bedd1-561d-4829-a689-2fa097f4a085 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:25.567516 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 207/420] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:24:25.573098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d37d7-c969-4808-8d1a-d00e583be9cb None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:25.573364 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d37d7-c969-4808-8d1a-d00e583be9cb None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:25.573629 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d37d7-c969-4808-8d1a-d00e583be9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:25.598753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-3e7d37d7-c969-4808-8d1a-d00e583be9cb None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:25.599938 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3e7d37d7-c969-4808-8d1a-d00e583be9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:25.621943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef4f1e-4105-4629-bb21-f57e54154693 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:25.623405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef4f1e-4105-4629-bb21-f57e54154693 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:25.666577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:25.732297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 208/421] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:24:25.732514 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:25.808109 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 214/422] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:25.814226 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d37d7-c969-4808-8d1a-d00e583be9cb None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:25.976968 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d37d7-c969-4808-8d1a-d00e583be9cb None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:26.019680 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41b83f-8cf5-416a-b422-b6893469e94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:26.020425 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41b83f-8cf5-416a-b422-b6893469e94e None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:26.020704 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41b83f-8cf5-416a-b422-b6893469e94e None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:26.020973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41b83f-8cf5-416a-b422-b6893469e94e None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:26.022103 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da41b83f-8cf5-416a-b422-b6893469e94e None admin] RBAC: Authorizing `identity:get_project(project_id=09c1bb7fd3204dffae9576cd2681a4a3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:26.023641 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da41b83f-8cf5-416a-b422-b6893469e94e None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:26.026040 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 209/423] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:25 2022] GET /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:26.031589 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef4f1e-4105-4629-bb21-f57e54154693 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:26.072038 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41b83f-8cf5-416a-b422-b6893469e94e None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:26.153515 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:26.175175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef4f1e-4105-4629-bb21-f57e54154693 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:26.235330 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b77d3c-b6c9-4afe-8dc3-76f02cb11c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:26.235973 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b77d3c-b6c9-4afe-8dc3-76f02cb11c6d None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:26.236267 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b77d3c-b6c9-4afe-8dc3-76f02cb11c6d None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:26.236542 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b77d3c-b6c9-4afe-8dc3-76f02cb11c6d None admin] PATH_INFO: `/v3/groups/9989165196cc40a19882bb7d70ac92e3` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:26.241469 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b77d3c-b6c9-4afe-8dc3-76f02cb11c6d None admin] RBAC: Authorizing `identity:get_group(group_id=9989165196cc40a19882bb7d70ac92e3)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:26.244927 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41b83f-8cf5-416a-b422-b6893469e94e None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:26.246886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b77d3c-b6c9-4afe-8dc3-76f02cb11c6d None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:26.248854 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 215/424] 10.222.0.59 () {56 vars in 1180 bytes} [Mon Aug 29 22:24:26 2022] GET /identity/v3/groups/9989165196cc40a19882bb7d70ac92e3 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:26.257107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b77d3c-b6c9-4afe-8dc3-76f02cb11c6d None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:26.291941 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94d180-8f1a-482d-a2c9-26de28702bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:26.293008 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94d180-8f1a-482d-a2c9-26de28702bc3 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:26.293509 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94d180-8f1a-482d-a2c9-26de28702bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:26.293979 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94d180-8f1a-482d-a2c9-26de28702bc3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:26.315177 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94d180-8f1a-482d-a2c9-26de28702bc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:26.324677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-4a94d180-8f1a-482d-a2c9-26de28702bc3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:26.324677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:26.324677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:26.324677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:26.324677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:26.324677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:26.324677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:26.324677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:26.324677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:26.324677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:26.324677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:26.324677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:26.324677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:26.324677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:26.324677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:26.324677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:26.325607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:26.325607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:26.325607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:26.325607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:26.325607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:26.325607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:26.325607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:26.325607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:26.325607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:26.325607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:26.325607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:26.325607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:26.325607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:26.325607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:26.325607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:26.325607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:26.326393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:26.326393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:26.326393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:26.326393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:26.326393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:26.326393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:26.326393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:26.326393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:26.326393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:26.326393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:26.326393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:26.326393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:26.328983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 210/425] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:26.332494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:26.338432 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94d180-8f1a-482d-a2c9-26de28702bc3 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:26.386769 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b77d3c-b6c9-4afe-8dc3-76f02cb11c6d None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:26.427838 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454e26c-6eab-46dc-b9c5-0d0ffc039e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:26.427838 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454e26c-6eab-46dc-b9c5-0d0ffc039e8e None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:26.427838 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454e26c-6eab-46dc-b9c5-0d0ffc039e8e None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:26.427838 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454e26c-6eab-46dc-b9c5-0d0ffc039e8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:26.428255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454e26c-6eab-46dc-b9c5-0d0ffc039e8e None admin] RBAC: Adding query filter params (group.id=9989165196cc40a19882bb7d70ac92e3, role.id=733cd39702ab4cbc860e34886dbb165f, scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:26.428255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454e26c-6eab-46dc-b9c5-0d0ffc039e8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:26.428255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454e26c-6eab-46dc-b9c5-0d0ffc039e8e None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:26.432461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 216/426] 10.222.0.59 () {56 vars in 1376 bytes} [Mon Aug 29 22:24:26 2022] GET /identity/v3/role_assignments?group.id=9989165196cc40a19882bb7d70ac92e3&scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3&role.id=733cd39702ab4cbc860e34886dbb165f => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:26.565470 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:26.568666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94d180-8f1a-482d-a2c9-26de28702bc3 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:26.615590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42bda08-84b6-44dd-aa03-e3542f00b83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:26.616251 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bda08-84b6-44dd-aa03-e3542f00b83e None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:26.616535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bda08-84b6-44dd-aa03-e3542f00b83e None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:26.616803 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bda08-84b6-44dd-aa03-e3542f00b83e None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:26.617455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42bda08-84b6-44dd-aa03-e3542f00b83e None admin] RBAC: Adding query filter params (name=member) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:26.617747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42bda08-84b6-44dd-aa03-e3542f00b83e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:26.619155 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42bda08-84b6-44dd-aa03-e3542f00b83e None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:26.628586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 211/427] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:24:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:26.633904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454e26c-6eab-46dc-b9c5-0d0ffc039e8e None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:26.759601 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:26.769675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454e26c-6eab-46dc-b9c5-0d0ffc039e8e None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:26.777601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03033e8b-2a38-47eb-8f2b-71fcbb9acc4f None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:26.777890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03033e8b-2a38-47eb-8f2b-71fcbb9acc4f None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:26.778159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03033e8b-2a38-47eb-8f2b-71fcbb9acc4f None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:26.779646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 212/428] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:26.786579 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d7592-d62d-4a8a-9bea-efa3393d95f8 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:26.786870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d7592-d62d-4a8a-9bea-efa3393d95f8 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:26.787149 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d7592-d62d-4a8a-9bea-efa3393d95f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:26.810351 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-3c0d7592-d62d-4a8a-9bea-efa3393d95f8 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:26.811714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3c0d7592-d62d-4a8a-9bea-efa3393d95f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:26.820285 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8d189-d6f1-4e91-9cfb-9455a240e7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:26.821233 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8d189-d6f1-4e91-9cfb-9455a240e7a2 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:26.821686 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8d189-d6f1-4e91-9cfb-9455a240e7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:26.822173 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8d189-d6f1-4e91-9cfb-9455a240e7a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:26.889671 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 213/429] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:26.909200 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8d189-d6f1-4e91-9cfb-9455a240e7a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:26.910951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8d189-d6f1-4e91-9cfb-9455a240e7a2 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:27.010202 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:27.020161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 217/430] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:27.026889 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d7592-d62d-4a8a-9bea-efa3393d95f8 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:27.133720 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:27.143059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d7592-d62d-4a8a-9bea-efa3393d95f8 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:27.179843 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09daa53-4d5f-455a-8143-afa672ac8b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:27.180535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09daa53-4d5f-455a-8143-afa672ac8b82 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:27.180784 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09daa53-4d5f-455a-8143-afa672ac8b82 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:27.181053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09daa53-4d5f-455a-8143-afa672ac8b82 None admin] PATH_INFO: `/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:27.182812 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09daa53-4d5f-455a-8143-afa672ac8b82 None admin] RBAC: Authorizing `identity:get_project(project_id=09c1bb7fd3204dffae9576cd2681a4a3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:27.184274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09daa53-4d5f-455a-8143-afa672ac8b82 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:27.186369 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 214/431] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:27 2022] GET /identity/v3/projects/09c1bb7fd3204dffae9576cd2681a4a3 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:27.190012 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8d189-d6f1-4e91-9cfb-9455a240e7a2 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:27.191783 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09daa53-4d5f-455a-8143-afa672ac8b82 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:27.319919 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8d189-d6f1-4e91-9cfb-9455a240e7a2 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:27.322156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09daa53-4d5f-455a-8143-afa672ac8b82 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:27.361135 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dc9c4-c42c-4899-9473-66f7080f7413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:27.361768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dc9c4-c42c-4899-9473-66f7080f7413 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:27.362145 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dc9c4-c42c-4899-9473-66f7080f7413 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:27.362419 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dc9c4-c42c-4899-9473-66f7080f7413 None admin] PATH_INFO: `/v3/groups/9989165196cc40a19882bb7d70ac92e3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:27.363547 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2dc9c4-c42c-4899-9473-66f7080f7413 None admin] RBAC: Authorizing `identity:get_group(group_id=9989165196cc40a19882bb7d70ac92e3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:27.365021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2dc9c4-c42c-4899-9473-66f7080f7413 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:27.366588 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 215/432] 10.222.0.59 () {56 vars in 1180 bytes} [Mon Aug 29 22:24:27 2022] GET /identity/v3/groups/9989165196cc40a19882bb7d70ac92e3 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:27.366696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9ad12-ee58-4be9-9f55-4f01dce7afb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:27.367300 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9ad12-ee58-4be9-9f55-4f01dce7afb4 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:27.367591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9ad12-ee58-4be9-9f55-4f01dce7afb4 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:27.367833 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9ad12-ee58-4be9-9f55-4f01dce7afb4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:27.372879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dc9c4-c42c-4899-9473-66f7080f7413 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:27.381215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9ad12-ee58-4be9-9f55-4f01dce7afb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:27.387614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-87e9ad12-ee58-4be9-9f55-4f01dce7afb4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:27.387614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:27.387614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:27.387614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:27.387614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:27.387614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:27.387614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:27.387614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:27.387614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:27.387614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:27.387614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:27.387614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:27.387614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:27.387614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:27.387614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:27.387614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:27.388535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:27.388535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:27.388535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:27.388535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:27.388535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:27.388535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:27.388535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:27.388535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:27.388535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:27.388535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:27.388535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:27.388535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:27.388535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:27.388535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:27.388535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:27.388535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:27.389457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:27.389457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:27.389457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:27.389457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:27.389457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:27.389457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:27.389457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:27.389457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:27.389457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:27.389457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:27.389457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 22:24:27.389457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:27.390069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 218/433] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:27.395341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9ad12-ee58-4be9-9f55-4f01dce7afb4 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:27.411796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:27.475290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:27.497896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dc9c4-c42c-4899-9473-66f7080f7413 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:27.529207 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9ad12-ee58-4be9-9f55-4f01dce7afb4 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:27.537528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cec3a4-0657-44b2-8cf1-643719c3d8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:27.538134 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cec3a4-0657-44b2-8cf1-643719c3d8dd None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:27.538410 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cec3a4-0657-44b2-8cf1-643719c3d8dd None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:27.538676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cec3a4-0657-44b2-8cf1-643719c3d8dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:27.539397 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cec3a4-0657-44b2-8cf1-643719c3d8dd None admin] RBAC: Adding query filter params (group.id=9989165196cc40a19882bb7d70ac92e3, role.id=454bd427dfc04030a4ec2b0e8a1ed237, scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:27.539694 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cec3a4-0657-44b2-8cf1-643719c3d8dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:27.541913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cec3a4-0657-44b2-8cf1-643719c3d8dd None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:27.556402 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 216/434] 10.222.0.59 () {56 vars in 1376 bytes} [Mon Aug 29 22:24:27 2022] GET /identity/v3/role_assignments?group.id=9989165196cc40a19882bb7d70ac92e3&scope.project.id=09c1bb7fd3204dffae9576cd2681a4a3&role.id=454bd427dfc04030a4ec2b0e8a1ed237 => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:27.558087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb7fd0-761f-419a-a54f-6ff8a2d22828 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:27.558347 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb7fd0-761f-419a-a54f-6ff8a2d22828 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:27.558615 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb7fd0-761f-419a-a54f-6ff8a2d22828 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:27.559951 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 217/435] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:27.565732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c41e8-aae8-4e8e-a01a-de3ae8f2a37d None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:27.566009 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c41e8-aae8-4e8e-a01a-de3ae8f2a37d None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:27.566283 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c41e8-aae8-4e8e-a01a-de3ae8f2a37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:27.581384 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48d0f5-2a91-4e21-853c-326641afbc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:27.582252 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48d0f5-2a91-4e21-853c-326641afbc38 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:27.582733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48d0f5-2a91-4e21-853c-326641afbc38 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:27.583147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48d0f5-2a91-4e21-853c-326641afbc38 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:27.584012 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48d0f5-2a91-4e21-853c-326641afbc38 None admin] RBAC: Adding query filter params (name=member) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:27.584511 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48d0f5-2a91-4e21-853c-326641afbc38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:27.586092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48d0f5-2a91-4e21-853c-326641afbc38 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:27.586224 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-135c41e8-aae8-4e8e-a01a-de3ae8f2a37d None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:27.587332 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-135c41e8-aae8-4e8e-a01a-de3ae8f2a37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:27.597172 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 219/436] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:24:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:27.602452 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48d0f5-2a91-4e21-853c-326641afbc38 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:27.686464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 218/437] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:24:27.688221 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6e495-f0b8-4ebd-a7c5-8bf2cbdf44e8 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:27.688515 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6e495-f0b8-4ebd-a7c5-8bf2cbdf44e8 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:27.688787 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6e495-f0b8-4ebd-a7c5-8bf2cbdf44e8 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:27.690529 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 219/438] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:27.696258 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898e9b0-9ae0-499f-8a03-edb131ea3618 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:27.696559 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898e9b0-9ae0-499f-8a03-edb131ea3618 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:27.697328 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898e9b0-9ae0-499f-8a03-edb131ea3618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:27.718114 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-b898e9b0-9ae0-499f-8a03-edb131ea3618 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:27.719508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b898e9b0-9ae0-499f-8a03-edb131ea3618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:27.738783 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:27.749898 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48d0f5-2a91-4e21-853c-326641afbc38 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:27.765935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:27.797276 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb855556-8877-4804-9218-647aa2d39e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:27.798348 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb855556-8877-4804-9218-647aa2d39e6e None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:27.799218 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb855556-8877-4804-9218-647aa2d39e6e None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:27.799649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb855556-8877-4804-9218-647aa2d39e6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:27.814954 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 220/439] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:24:27.884992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb855556-8877-4804-9218-647aa2d39e6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:27.886686 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb855556-8877-4804-9218-647aa2d39e6e None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:27.967302 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b898e9b0-9ae0-499f-8a03-edb131ea3618 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:27.993582 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 220/440] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 22:24:28.001212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb855556-8877-4804-9218-647aa2d39e6e None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:28.019802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:28.113477 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b898e9b0-9ae0-499f-8a03-edb131ea3618 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:28.151491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb855556-8877-4804-9218-647aa2d39e6e None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:28.156510 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0244bb-73aa-4a8b-b0ce-7f8b32ec95eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:28.157162 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0244bb-73aa-4a8b-b0ce-7f8b32ec95eb None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:28.157436 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0244bb-73aa-4a8b-b0ce-7f8b32ec95eb None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:28.157703 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0244bb-73aa-4a8b-b0ce-7f8b32ec95eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:28.173335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0244bb-73aa-4a8b-b0ce-7f8b32ec95eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:28.178413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-0b0244bb-73aa-4a8b-b0ce-7f8b32ec95eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:24:28.178413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:28.178413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:28.178413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:28.178413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:28.178413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:28.178413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:28.178413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:28.178413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:28.178413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:28.178413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:28.178413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:28.178413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:28.178413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:28.178413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:28.178413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:28.179374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:28.179374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:28.179374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:28.179374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:28.179374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:28.179374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:28.179374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:28.179374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:28.179374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:28.179374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:28.179374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:28.179374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:28.179374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:28.179374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:28.179374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:28.179374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:28.181210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:28.181210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:28.181210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:28.181210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:28.181210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:28.181210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:28.181210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:28.181210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:28.181210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:28.181210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:28.181210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:24:28.181210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:28.181210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 221/441] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:24:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:24:28.181210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0244bb-73aa-4a8b-b0ce-7f8b32ec95eb None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:28.195292 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d75616-0a87-4014-92bb-8b22ed1d5a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:28.195929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d75616-0a87-4014-92bb-8b22ed1d5a45 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:28.200610 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d75616-0a87-4014-92bb-8b22ed1d5a45 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:28.200610 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d75616-0a87-4014-92bb-8b22ed1d5a45 None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:28.200610 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d75616-0a87-4014-92bb-8b22ed1d5a45 None admin] RBAC: Authorizing `identity:get_project(project_id=7d4cfda3691c4a4ab82dda7546659db6)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:28.200610 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d75616-0a87-4014-92bb-8b22ed1d5a45 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:28.200919 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:28.202896 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 221/442] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:28 2022] GET /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:28.203025 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d75616-0a87-4014-92bb-8b22ed1d5a45 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:28.290414 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:28.300735 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0244bb-73aa-4a8b-b0ce-7f8b32ec95eb None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:28.309589 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d75616-0a87-4014-92bb-8b22ed1d5a45 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:28.335201 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b9f36-9463-4a44-8499-f4240042ba95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:28.335828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b9f36-9463-4a44-8499-f4240042ba95 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:28.336104 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b9f36-9463-4a44-8499-f4240042ba95 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:28.336417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b9f36-9463-4a44-8499-f4240042ba95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:28.344645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397cd8e-4746-46fc-9fc5-0f578f5d4cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:28.345238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397cd8e-4746-46fc-9fc5-0f578f5d4cc1 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:28.345508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397cd8e-4746-46fc-9fc5-0f578f5d4cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:28.345769 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397cd8e-4746-46fc-9fc5-0f578f5d4cc1 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:28.346437 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397cd8e-4746-46fc-9fc5-0f578f5d4cc1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:28.346729 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397cd8e-4746-46fc-9fc5-0f578f5d4cc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:28.348072 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397cd8e-4746-46fc-9fc5-0f578f5d4cc1 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:28.348500 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b9f36-9463-4a44-8499-f4240042ba95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:28.354306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-7a2b9f36-9463-4a44-8499-f4240042ba95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:24:28.354306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:28.354306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:28.354306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:28.354306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:28.354306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:28.354306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:28.354306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:28.354306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:28.354306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:28.354306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:28.354306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:28.354306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:28.354306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:28.354306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:28.354306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:28.355256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:28.355256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:28.355256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:28.355256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:28.355256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:28.355256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:28.355256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:28.355256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:28.355256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:28.355256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:28.355256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:28.355256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:28.355256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:28.355256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:28.355256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:28.355256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:28.356067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:28.356067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:28.356067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:28.356067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:28.356067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:28.356067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:28.356067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:28.356067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:28.356067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:28.356067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:28.356067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:24:28.356067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:28.356067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 222/443] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:24:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:28.356785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b9f36-9463-4a44-8499-f4240042ba95 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:28.356851 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 222/444] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:28.361193 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397cd8e-4746-46fc-9fc5-0f578f5d4cc1 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:28.429277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:28.459619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b9f36-9463-4a44-8499-f4240042ba95 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:28.464276 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397cd8e-4746-46fc-9fc5-0f578f5d4cc1 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:28.499175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3decd6-480c-4ddb-a249-aa50a91e8b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:28.499391 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee9aee-3826-44e6-bf43-dcdd872c1d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:28.499772 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3decd6-480c-4ddb-a249-aa50a91e8b56 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:28.499867 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee9aee-3826-44e6-bf43-dcdd872c1d76 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:28.500042 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3decd6-480c-4ddb-a249-aa50a91e8b56 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:28.500130 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee9aee-3826-44e6-bf43-dcdd872c1d76 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:28.500335 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3decd6-480c-4ddb-a249-aa50a91e8b56 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:28.500437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee9aee-3826-44e6-bf43-dcdd872c1d76 None admin] PATH_INFO: `/v3/groups/9989165196cc40a19882bb7d70ac92e3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:28.500986 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3decd6-480c-4ddb-a249-aa50a91e8b56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:28.501280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3decd6-480c-4ddb-a249-aa50a91e8b56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:28.501492 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee9aee-3826-44e6-bf43-dcdd872c1d76 None admin] RBAC: Authorizing `identity:get_group(group_id=9989165196cc40a19882bb7d70ac92e3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:28.502637 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3decd6-480c-4ddb-a249-aa50a91e8b56 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:28.502922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee9aee-3826-44e6-bf43-dcdd872c1d76 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:28.504501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 223/445] 10.222.0.59 () {56 vars in 1180 bytes} [Mon Aug 29 22:24:28 2022] GET /identity/v3/groups/9989165196cc40a19882bb7d70ac92e3 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:28.505257 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee9aee-3826-44e6-bf43-dcdd872c1d76 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:28.512486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 223/446] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:28.513312 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3decd6-480c-4ddb-a249-aa50a91e8b56 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:28.522997 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:28.616084 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3decd6-480c-4ddb-a249-aa50a91e8b56 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:28.618383 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee9aee-3826-44e6-bf43-dcdd872c1d76 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:28.651687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddaad3c-dccd-4140-be67-a926f1d1b070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:28.652381 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddaad3c-dccd-4140-be67-a926f1d1b070 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:28.652629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddaad3c-dccd-4140-be67-a926f1d1b070 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:28.652901 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddaad3c-dccd-4140-be67-a926f1d1b070 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:28.653623 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddaad3c-dccd-4140-be67-a926f1d1b070 None admin] RBAC: Adding query filter params (group.id=9989165196cc40a19882bb7d70ac92e3, role.id=454bd427dfc04030a4ec2b0e8a1ed237, scope.project.id=7d4cfda3691c4a4ab82dda7546659db6) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:28.653929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddaad3c-dccd-4140-be67-a926f1d1b070 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:28.654484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca4970-eda4-4fc7-ad9e-fc716c9b6ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:28.655082 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca4970-eda4-4fc7-ad9e-fc716c9b6ad0 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:28.655354 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca4970-eda4-4fc7-ad9e-fc716c9b6ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:28.655441 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddaad3c-dccd-4140-be67-a926f1d1b070 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:28.655618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca4970-eda4-4fc7-ad9e-fc716c9b6ad0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:28.661464 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 224/447] 10.222.0.59 () {56 vars in 1376 bytes} [Mon Aug 29 22:24:28 2022] GET /identity/v3/role_assignments?group.id=9989165196cc40a19882bb7d70ac92e3&scope.project.id=7d4cfda3691c4a4ab82dda7546659db6&role.id=454bd427dfc04030a4ec2b0e8a1ed237 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:28.662355 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddaad3c-dccd-4140-be67-a926f1d1b070 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:28.667761 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:28.733959 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca4970-eda4-4fc7-ad9e-fc716c9b6ad0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:28.735499 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca4970-eda4-4fc7-ad9e-fc716c9b6ad0 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:28.758711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:28.796798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddaad3c-dccd-4140-be67-a926f1d1b070 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:28.841526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd37280-cbb1-49ba-8d93-5eca525b8504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:28.842143 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd37280-cbb1-49ba-8d93-5eca525b8504 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:28.842417 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd37280-cbb1-49ba-8d93-5eca525b8504 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:28.842689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd37280-cbb1-49ba-8d93-5eca525b8504 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:28.856957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 224/448] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:28.863113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca4970-eda4-4fc7-ad9e-fc716c9b6ad0 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:28.914829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:28.936254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd37280-cbb1-49ba-8d93-5eca525b8504 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:28.937790 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd37280-cbb1-49ba-8d93-5eca525b8504 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:28.991618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca4970-eda4-4fc7-ad9e-fc716c9b6ad0 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:29.024066 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:29.035814 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702878f-f44e-4db6-a55b-cbd6380ee4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:29.036478 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702878f-f44e-4db6-a55b-cbd6380ee4d4 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:29.036746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702878f-f44e-4db6-a55b-cbd6380ee4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:29.037012 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702878f-f44e-4db6-a55b-cbd6380ee4d4 None admin] PATH_INFO: `/v3/projects/7d4cfda3691c4a4ab82dda7546659db6` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:29.038120 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702878f-f44e-4db6-a55b-cbd6380ee4d4 None admin] RBAC: Authorizing `identity:get_project(project_id=7d4cfda3691c4a4ab82dda7546659db6)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:29.039556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702878f-f44e-4db6-a55b-cbd6380ee4d4 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:29.042136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 225/449] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:28 2022] GET /identity/v3/projects/7d4cfda3691c4a4ab82dda7546659db6 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:29.047529 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702878f-f44e-4db6-a55b-cbd6380ee4d4 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:29.056844 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 225/450] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:29.062815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd37280-cbb1-49ba-8d93-5eca525b8504 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:29.152045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702878f-f44e-4db6-a55b-cbd6380ee4d4 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:29.161535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd37280-cbb1-49ba-8d93-5eca525b8504 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:29.161714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:29.188028 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ddb06b-1f4d-4a62-a722-83f9b2c84ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:29.188775 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddb06b-1f4d-4a62-a722-83f9b2c84ebc None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:29.189238 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddb06b-1f4d-4a62-a722-83f9b2c84ebc None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:29.189694 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddb06b-1f4d-4a62-a722-83f9b2c84ebc None admin] PATH_INFO: `/v3/groups/9989165196cc40a19882bb7d70ac92e3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:29.191390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ddb06b-1f4d-4a62-a722-83f9b2c84ebc None admin] RBAC: Authorizing `identity:get_group(group_id=9989165196cc40a19882bb7d70ac92e3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:29.192856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ddb06b-1f4d-4a62-a722-83f9b2c84ebc None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:29.194678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 226/451] 10.222.0.59 () {56 vars in 1180 bytes} [Mon Aug 29 22:24:29 2022] GET /identity/v3/groups/9989165196cc40a19882bb7d70ac92e3 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:29.198736 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4c78e-45b4-4a47-b21a-a4d69a61be55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:29.199208 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4c78e-45b4-4a47-b21a-a4d69a61be55 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:29.199491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4c78e-45b4-4a47-b21a-a4d69a61be55 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:29.199814 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4c78e-45b4-4a47-b21a-a4d69a61be55 None admin] PATH_INFO: `/v3/projects/d3d78b58a69d4b10868c64742da904fe` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:29.200566 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ddb06b-1f4d-4a62-a722-83f9b2c84ebc None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:29.201194 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4c78e-45b4-4a47-b21a-a4d69a61be55 None admin] RBAC: Authorizing `identity:get_project(project_id=d3d78b58a69d4b10868c64742da904fe)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:29.202506 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4c78e-45b4-4a47-b21a-a4d69a61be55 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:29.204853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 226/452] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:24:29 2022] GET /identity/v3/projects/d3d78b58a69d4b10868c64742da904fe => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:29.210464 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4c78e-45b4-4a47-b21a-a4d69a61be55 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:29.254746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:29.315827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ddb06b-1f4d-4a62-a722-83f9b2c84ebc None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:29.322680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4c78e-45b4-4a47-b21a-a4d69a61be55 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:29.350673 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8dbe7-b7d2-4ed2-a736-7cbcd484bd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:29.351267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8dbe7-b7d2-4ed2-a736-7cbcd484bd84 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:29.351543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8dbe7-b7d2-4ed2-a736-7cbcd484bd84 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:29.352007 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8dbe7-b7d2-4ed2-a736-7cbcd484bd84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:29.352618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8dbe7-b7d2-4ed2-a736-7cbcd484bd84 None admin] RBAC: Adding query filter params (group.id=9989165196cc40a19882bb7d70ac92e3, role.id=733cd39702ab4cbc860e34886dbb165f, scope.project.id=7d4cfda3691c4a4ab82dda7546659db6) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:29.352834 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8dbe7-b7d2-4ed2-a736-7cbcd484bd84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:29.354907 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8dbe7-b7d2-4ed2-a736-7cbcd484bd84 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:29.357321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f606798-86ca-4df7-a505-0c8e36bc0420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:29.358038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f606798-86ca-4df7-a505-0c8e36bc0420 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:29.358324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f606798-86ca-4df7-a505-0c8e36bc0420 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:29.358593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f606798-86ca-4df7-a505-0c8e36bc0420 None admin] PATH_INFO: `/v3/groups/58bc1551bf5a4d2e9bf9e41461a32198` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:29.359708 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f606798-86ca-4df7-a505-0c8e36bc0420 None admin] RBAC: Authorizing `identity:get_group(group_id=58bc1551bf5a4d2e9bf9e41461a32198)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:29.360750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 227/453] 10.222.0.59 () {56 vars in 1376 bytes} [Mon Aug 29 22:24:29 2022] GET /identity/v3/role_assignments?group.id=9989165196cc40a19882bb7d70ac92e3&scope.project.id=7d4cfda3691c4a4ab82dda7546659db6&role.id=733cd39702ab4cbc860e34886dbb165f => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:29.361152 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f606798-86ca-4df7-a505-0c8e36bc0420 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:29.362811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 227/454] 10.222.0.59 () {56 vars in 1180 bytes} [Mon Aug 29 22:24:29 2022] GET /identity/v3/groups/58bc1551bf5a4d2e9bf9e41461a32198 => generated 229 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:29.369252 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8dbe7-b7d2-4ed2-a736-7cbcd484bd84 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:29.416547 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:29.479598 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8dbe7-b7d2-4ed2-a736-7cbcd484bd84 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:29.516590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a944d-c1cc-41e6-9910-876737fb53cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:29.518070 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a944d-c1cc-41e6-9910-876737fb53cb None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:29.518353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a944d-c1cc-41e6-9910-876737fb53cb None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:29.518627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a944d-c1cc-41e6-9910-876737fb53cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:29.519362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a944d-c1cc-41e6-9910-876737fb53cb None admin] RBAC: Adding query filter params (group.id=58bc1551bf5a4d2e9bf9e41461a32198, role.id=a836108447dc439e994611bc74d53d9c, scope.project.id=d3d78b58a69d4b10868c64742da904fe) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:29.519687 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a944d-c1cc-41e6-9910-876737fb53cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:29.521175 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a944d-c1cc-41e6-9910-876737fb53cb None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:29.528214 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 228/455] 10.222.0.59 () {56 vars in 1376 bytes} [Mon Aug 29 22:24:29 2022] GET /identity/v3/role_assignments?group.id=58bc1551bf5a4d2e9bf9e41461a32198&scope.project.id=d3d78b58a69d4b10868c64742da904fe&role.id=a836108447dc439e994611bc74d53d9c => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:31.859320 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fce5df-c6a5-4335-81e6-d30716b51b2a None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:31.860366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fce5df-c6a5-4335-81e6-d30716b51b2a None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:31.860847 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fce5df-c6a5-4335-81e6-d30716b51b2a None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:31.862649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 228/456] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:31.868624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4fbb1-ab86-4024-863b-54cbcb43a186 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:31.869750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4fbb1-ab86-4024-863b-54cbcb43a186 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:31.870380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4fbb1-ab86-4024-863b-54cbcb43a186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:31.892093 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-b2f4fbb1-ab86-4024-863b-54cbcb43a186 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:31.893662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b2f4fbb1-ab86-4024-863b-54cbcb43a186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:32.000299 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 229/457] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:32.101498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147f48f-5793-44a9-bf7d-810624e18159 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:32.102123 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147f48f-5793-44a9-bf7d-810624e18159 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:32.102602 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147f48f-5793-44a9-bf7d-810624e18159 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:32.105303 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 229/458] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:32.111630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbed6b-9906-4fd3-8b1b-d2a987dd0e0e None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:32.112202 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbed6b-9906-4fd3-8b1b-d2a987dd0e0e None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:32.112669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbed6b-9906-4fd3-8b1b-d2a987dd0e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:32.135452 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-8ffbed6b-9906-4fd3-8b1b-d2a987dd0e0e None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:32.137114 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8ffbed6b-9906-4fd3-8b1b-d2a987dd0e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:32.174689 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:32.232194 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 230/459] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:32.257443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076baad-3fa3-43f5-853d-5d45eaad9a58 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:32.258032 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076baad-3fa3-43f5-853d-5d45eaad9a58 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:32.258503 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076baad-3fa3-43f5-853d-5d45eaad9a58 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:32.260917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 230/460] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:32.266475 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8478b45-5644-415a-8d29-61e5ae47f527 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:32.266755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8478b45-5644-415a-8d29-61e5ae47f527 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:32.267029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8478b45-5644-415a-8d29-61e5ae47f527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:32.294245 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-e8478b45-5644-415a-8d29-61e5ae47f527 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:32.295417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e8478b45-5644-415a-8d29-61e5ae47f527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:32.349885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076baad-3fa3-43f5-853d-5d45eaad9a58 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:32.403001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 231/461] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:32.505770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076baad-3fa3-43f5-853d-5d45eaad9a58 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:32.515148 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:32.558557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccbf42-15d1-4d65-b98e-e4b7d0d1ebb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:32.559242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccbf42-15d1-4d65-b98e-e4b7d0d1ebb7 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:32.559514 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccbf42-15d1-4d65-b98e-e4b7d0d1ebb7 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:32.559778 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccbf42-15d1-4d65-b98e-e4b7d0d1ebb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:32.574518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ccbf42-15d1-4d65-b98e-e4b7d0d1ebb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:32.575960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ccbf42-15d1-4d65-b98e-e4b7d0d1ebb7 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:32.591554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-a8ccbf42-15d1-4d65-b98e-e4b7d0d1ebb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:32.591554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:32.591554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:32.591554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:32.591554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:32.591554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:32.591554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:32.591554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:32.591554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:32.591554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:32.591554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:32.591554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:32.591554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:32.591554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:32.591554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:32.591554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:32.592532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:32.592532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:32.592532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:32.592532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:32.592532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:32.592532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:32.592532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:32.592532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:32.592532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:32.592532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:32.592532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:32.592532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:32.592532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:32.592532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:32.592532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:32.592532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:32.592532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:32.593383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:32.593383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:32.593383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:32.597054 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 231/462] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:24:32.602621 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8478b45-5644-415a-8d29-61e5ae47f527 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:32.608626 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccbf42-15d1-4d65-b98e-e4b7d0d1ebb7 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:32.782394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:32.791065 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8478b45-5644-415a-8d29-61e5ae47f527 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:32.792194 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccbf42-15d1-4d65-b98e-e4b7d0d1ebb7 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:32.834441 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7232d7f-e311-46f8-bfe0-082b460f22b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:32.835069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7232d7f-e311-46f8-bfe0-082b460f22b4 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:32.835346 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7232d7f-e311-46f8-bfe0-082b460f22b4 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:32.835616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7232d7f-e311-46f8-bfe0-082b460f22b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:32.836642 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e14f6-7a2a-4373-831f-d47e681c93cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:32.837926 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e14f6-7a2a-4373-831f-d47e681c93cb None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:32.838434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e14f6-7a2a-4373-831f-d47e681c93cb None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:32.838994 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e14f6-7a2a-4373-831f-d47e681c93cb None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:32.839848 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e14f6-7a2a-4373-831f-d47e681c93cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:32.840388 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e14f6-7a2a-4373-831f-d47e681c93cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:32.842001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e14f6-7a2a-4373-831f-d47e681c93cb None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:32.848106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7232d7f-e311-46f8-bfe0-082b460f22b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:32.849627 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7232d7f-e311-46f8-bfe0-082b460f22b4 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:32.856468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 232/463] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:32.857695 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e14f6-7a2a-4373-831f-d47e681c93cb None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:32.867488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-d7232d7f-e311-46f8-bfe0-082b460f22b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:32.867488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:32.867488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:32.867488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:32.867488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:32.867488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:32.867488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:32.867488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:32.867488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:32.867488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:32.867488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:32.867488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:32.867488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:32.867488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:32.867488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:32.867488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:32.869746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:32.869746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:32.869746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:32.869746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:32.869746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:32.869746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:32.869746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:32.869746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:32.869746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:32.869746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:32.869746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:32.869746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:32.869746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:32.869746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:32.869746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:32.869746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:32.869746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:32.875764 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:32.875764 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:32.875764 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:32.875764 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 232/464] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:32.875764 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7232d7f-e311-46f8-bfe0-082b460f22b4 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:32.890728 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:32.985695 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7232d7f-e311-46f8-bfe0-082b460f22b4 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:32.989157 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e14f6-7a2a-4373-831f-d47e681c93cb None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:33.023905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd1b45-2233-4a8a-a5ae-a51dceba2458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:33.024426 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf93a69-d1ef-4d59-a1d3-dce3707cc5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:33.024582 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd1b45-2233-4a8a-a5ae-a51dceba2458 None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:33.025497 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd1b45-2233-4a8a-a5ae-a51dceba2458 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:33.025776 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd1b45-2233-4a8a-a5ae-a51dceba2458 None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:33.025928 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf93a69-d1ef-4d59-a1d3-dce3707cc5cd None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:33.026387 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf93a69-d1ef-4d59-a1d3-dce3707cc5cd None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:33.026782 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd1b45-2233-4a8a-a5ae-a51dceba2458 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:33.026874 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf93a69-d1ef-4d59-a1d3-dce3707cc5cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:33.028389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd1b45-2233-4a8a-a5ae-a51dceba2458 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:33.038485 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf93a69-d1ef-4d59-a1d3-dce3707cc5cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:33.039844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf93a69-d1ef-4d59-a1d3-dce3707cc5cd None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:33.055156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-fdf93a69-d1ef-4d59-a1d3-dce3707cc5cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:33.055156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:33.055156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:33.055156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:33.055156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:33.055156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:33.055156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:33.055156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:33.055156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:33.055156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:33.055156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:33.055156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:33.055156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:33.055156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:33.055156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:33.055156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:33.061190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:33.061190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:33.061190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:33.061190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:33.061190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:33.061190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:33.061190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:33.061190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:33.061190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:33.061190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:33.061190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:33.061190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:33.061190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:33.061190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:33.061190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:33.061190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:33.061190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:33.062175 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:33.062175 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:33.062175 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:33.062175 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 233/465] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:33.062175 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf93a69-d1ef-4d59-a1d3-dce3707cc5cd None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:33.067611 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:33.139110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 233/466] 10.222.0.59 () {60 vars in 1132 bytes} [Mon Aug 29 22:24:32 2022] POST /identity/v3/users => generated 246 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:24:33.140728 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1547d-e694-44f1-937e-3464bb37abac None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:33.141015 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1547d-e694-44f1-937e-3464bb37abac None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:33.141292 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1547d-e694-44f1-937e-3464bb37abac None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:33.142667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 234/467] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:33.147704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1547d-e694-44f1-937e-3464bb37abac None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:33.184609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf93a69-d1ef-4d59-a1d3-dce3707cc5cd None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:33.228547 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c069d-6ead-4e77-a71d-01b5c6490da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:33.229832 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c069d-6ead-4e77-a71d-01b5c6490da6 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:33.230441 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c069d-6ead-4e77-a71d-01b5c6490da6 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:33.230997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c069d-6ead-4e77-a71d-01b5c6490da6 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:33.231861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c069d-6ead-4e77-a71d-01b5c6490da6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:33.232394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c069d-6ead-4e77-a71d-01b5c6490da6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:33.232867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:33.235481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c069d-6ead-4e77-a71d-01b5c6490da6 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:33.250491 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 234/468] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:33.252370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7ec07-a59b-4026-a57d-deda75acad29 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:33.252853 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7ec07-a59b-4026-a57d-deda75acad29 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:33.253286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7ec07-a59b-4026-a57d-deda75acad29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:33.274319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-b7b7ec07-a59b-4026-a57d-deda75acad29 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:33.277967 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1547d-e694-44f1-937e-3464bb37abac None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:33.278999 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b7b7ec07-a59b-4026-a57d-deda75acad29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:33.324023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00239ce0-f7bb-4652-9059-bc6f2885daa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:33.324602 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00239ce0-f7bb-4652-9059-bc6f2885daa2 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:33.324881 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00239ce0-f7bb-4652-9059-bc6f2885daa2 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:33.325179 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00239ce0-f7bb-4652-9059-bc6f2885daa2 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:33.325836 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00239ce0-f7bb-4652-9059-bc6f2885daa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:33.326133 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00239ce0-f7bb-4652-9059-bc6f2885daa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:33.327506 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00239ce0-f7bb-4652-9059-bc6f2885daa2 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:33.341727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 235/469] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:33.343197 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00239ce0-f7bb-4652-9059-bc6f2885daa2 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:33.356050 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:33.366154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 235/470] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:33.370426 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7ec07-a59b-4026-a57d-deda75acad29 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:33.459098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00239ce0-f7bb-4652-9059-bc6f2885daa2 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:33.481161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7ec07-a59b-4026-a57d-deda75acad29 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:33.490498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:33.502471 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3f670-4249-4dd0-9c0b-26b9ebb0b8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:33.503677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3f670-4249-4dd0-9c0b-26b9ebb0b8fa None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:33.503677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3f670-4249-4dd0-9c0b-26b9ebb0b8fa None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:33.503677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3f670-4249-4dd0-9c0b-26b9ebb0b8fa None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:33.504443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3f670-4249-4dd0-9c0b-26b9ebb0b8fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:33.506105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3f670-4249-4dd0-9c0b-26b9ebb0b8fa None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:33.524455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17547f07-2bd8-4689-a356-d97d45f09a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:33.525965 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17547f07-2bd8-4689-a356-d97d45f09a7e None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:33.526271 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17547f07-2bd8-4689-a356-d97d45f09a7e None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:33.526503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17547f07-2bd8-4689-a356-d97d45f09a7e None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:33.527144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17547f07-2bd8-4689-a356-d97d45f09a7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:33.528594 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17547f07-2bd8-4689-a356-d97d45f09a7e None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:33.609097 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 236/471] 10.222.0.59 () {60 vars in 1132 bytes} [Mon Aug 29 22:24:33 2022] POST /identity/v3/users => generated 248 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:24:33.626694 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 236/472] 10.222.0.59 () {60 vars in 1132 bytes} [Mon Aug 29 22:24:33 2022] POST /identity/v3/users => generated 248 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:24:33.670406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3f670-4249-4dd0-9c0b-26b9ebb0b8fa None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:33.690967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968be4f-b380-49a4-af50-a949334d32ed None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:33.691267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968be4f-b380-49a4-af50-a949334d32ed None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:33.691680 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968be4f-b380-49a4-af50-a949334d32ed None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:33.697632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 237/473] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:33.699668 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a6ccd-7037-47bb-94f5-d8fbdc02c799 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:33.700023 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a6ccd-7037-47bb-94f5-d8fbdc02c799 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:33.700413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a6ccd-7037-47bb-94f5-d8fbdc02c799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:33.720827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-1b9a6ccd-7037-47bb-94f5-d8fbdc02c799 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:33.722190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-1b9a6ccd-7037-47bb-94f5-d8fbdc02c799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:33.801834 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:33.829017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 238/474] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:33.864110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3f670-4249-4dd0-9c0b-26b9ebb0b8fa None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:33.904264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6044f-ee4d-492e-b1d5-c034916958a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:33.905685 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6044f-ee4d-492e-b1d5-c034916958a6 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:33.906207 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6044f-ee4d-492e-b1d5-c034916958a6 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:33.906685 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6044f-ee4d-492e-b1d5-c034916958a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:33.927184 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6044f-ee4d-492e-b1d5-c034916958a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:33.929015 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6044f-ee4d-492e-b1d5-c034916958a6 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:33.945146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-55d6044f-ee4d-492e-b1d5-c034916958a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:33.945146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:33.945146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:33.945146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:33.945146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:33.945146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:33.945146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:33.945146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:33.945146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:33.945146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:33.945146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:33.945146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:33.945146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:33.945146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:33.945146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:33.945146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:33.946094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:33.946094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:33.946094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:33.946094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:33.946094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:33.946094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:33.946094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:33.946094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:33.946094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:33.946094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:33.946094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:33.946094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:33.946094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:33.946094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:33.946094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:33.946094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:33.946094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:33.947464 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:33.947464 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:33.947464 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:33.949901 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 237/475] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:33.955754 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a6ccd-7037-47bb-94f5-d8fbdc02c799 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:34.071172 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a6ccd-7037-47bb-94f5-d8fbdc02c799 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:34.130207 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f2fc2-119d-40f5-9835-d107f5a26cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:34.130818 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f2fc2-119d-40f5-9835-d107f5a26cb7 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:34.131088 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f2fc2-119d-40f5-9835-d107f5a26cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:34.131355 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f2fc2-119d-40f5-9835-d107f5a26cb7 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:34.132006 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f2fc2-119d-40f5-9835-d107f5a26cb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:34.132328 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f2fc2-119d-40f5-9835-d107f5a26cb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:34.133697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f2fc2-119d-40f5-9835-d107f5a26cb7 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:34.154319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 239/476] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:34.160028 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6044f-ee4d-492e-b1d5-c034916958a6 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:34.164208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f2fc2-119d-40f5-9835-d107f5a26cb7 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:34.216609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:34.242600 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:34.309466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6044f-ee4d-492e-b1d5-c034916958a6 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:34.314563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f2fc2-119d-40f5-9835-d107f5a26cb7 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:34.348122 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352f3ce-56ad-4b58-9596-cfea9a583181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:34.348757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352f3ce-56ad-4b58-9596-cfea9a583181 None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:34.349029 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352f3ce-56ad-4b58-9596-cfea9a583181 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:34.349293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352f3ce-56ad-4b58-9596-cfea9a583181 None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:34.349949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352f3ce-56ad-4b58-9596-cfea9a583181 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:34.351365 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352f3ce-56ad-4b58-9596-cfea9a583181 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:34.368538 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b82d7-5c6c-4f87-900a-ed147d36befa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:34.369449 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b82d7-5c6c-4f87-900a-ed147d36befa None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:34.369725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b82d7-5c6c-4f87-900a-ed147d36befa None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:34.369993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b82d7-5c6c-4f87-900a-ed147d36befa None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:34.370672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b82d7-5c6c-4f87-900a-ed147d36befa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:34.372085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b82d7-5c6c-4f87-900a-ed147d36befa None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:34.418526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.sql.core [None req-069b82d7-5c6c-4f87-900a-ed147d36befa None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 22:24:34.418526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 22:24:34.418526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [parameters: {'id': 'e741161b5a904215a81debb09b49523d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 22:24:34.418526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=82165) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 22:24:34.436119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-069b82d7-5c6c-4f87-900a-ed147d36befa None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 22:24:34.436119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:34.436119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 22:24:34.436119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 22:24:34.436119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 22:24:34.436119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 22:24:34.436119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 22:24:34.436119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 22:24:34.436119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 22:24:34.436119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application conn.query(q) Aug 29 22:24:34.436119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 22:24:34.436119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 22:24:34.436119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 22:24:34.436119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application result.read() Aug 29 22:24:34.436119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 22:24:34.437045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 22:24:34.437045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 22:24:34.437045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 22:24:34.437045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 22:24:34.437045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 22:24:34.437045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 22:24:34.437045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 22:24:34.437045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 22:24:34.437045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:34.437045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 22:24:34.437045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:34.437045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:34.437045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 22:24:34.437045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 22:24:34.437045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 29 22:24:34.437045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return ref.to_dict() Aug 29 22:24:34.437045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 22:24:34.437045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application next(self.gen) Aug 29 22:24:34.437931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 22:24:34.437931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application yield resource Aug 29 22:24:34.437931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 22:24:34.437931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application next(self.gen) Aug 29 22:24:34.437931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 22:24:34.437931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 22:24:34.437931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 22:24:34.437931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 22:24:34.437931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 22:24:34.437931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise self.value Aug 29 22:24:34.437931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 22:24:34.437931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 22:24:34.437931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 22:24:34.437931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application session.commit() Aug 29 22:24:34.437931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 22:24:34.437931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 22:24:34.437931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 22:24:34.439676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 22:24:34.439676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 22:24:34.439676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self.session.flush() Aug 29 22:24:34.439676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 22:24:34.439676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 22:24:34.439676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 22:24:34.439676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 22:24:34.439676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 22:24:34.439676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application compat.raise_( Aug 29 22:24:34.439676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 22:24:34.439676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception Aug 29 22:24:34.439676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 22:24:34.439676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 22:24:34.439676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 22:24:34.439676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 22:24:34.439676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 22:24:34.439676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 22:24:34.441769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 22:24:34.441769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 22:24:34.441769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 22:24:34.441769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 22:24:34.441769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 22:24:34.441769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 22:24:34.441769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 22:24:34.441769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 22:24:34.441769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 22:24:34.441769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 22:24:34.441769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 22:24:34.441769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 22:24:34.441769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 22:24:34.441769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 22:24:34.441769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 22:24:34.441769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception Aug 29 22:24:34.443886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 22:24:34.443886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 22:24:34.443886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 22:24:34.443886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 22:24:34.443886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 22:24:34.443886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 22:24:34.443886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 22:24:34.443886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application conn.query(q) Aug 29 22:24:34.443886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 22:24:34.443886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 22:24:34.443886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 22:24:34.443886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application result.read() Aug 29 22:24:34.443886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 22:24:34.443886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 22:24:34.443886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 22:24:34.443886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 22:24:34.446854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 22:24:34.446854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 22:24:34.446854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 22:24:34.446854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 22:24:34.446854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 22:24:34.446854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 22:24:34.446854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [parameters: {'id': 'e741161b5a904215a81debb09b49523d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 22:24:34.446854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 22:24:34.446854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:34.446854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:24:34.446854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:34.446854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:34.446854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:34.446854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:34.446854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:34.446854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:34.449035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:34.449035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:34.449035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:34.449035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:34.449035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:34.449035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:34.449035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 29 22:24:34.449035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 29 22:24:34.449035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:34.449035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:34.449035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 29 22:24:34.449035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 29 22:24:34.449035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 22:24:34.449035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 22:24:34.449035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 22:24:34.449035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:34.451213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 240/477] 10.222.0.59 () {60 vars in 1131 bytes} [Mon Aug 29 22:24:34 2022] POST /identity/v3/roles => generated 144 bytes in 274 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:24:34.451213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b82d7-5c6c-4f87-900a-ed147d36befa None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:34.466668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 238/478] 10.222.0.59 () {60 vars in 1132 bytes} [Mon Aug 29 22:24:34 2022] POST /identity/v3/users => generated 249 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:24:34.571933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b82d7-5c6c-4f87-900a-ed147d36befa None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:34.580908 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:34.621641 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc2b2b-d13c-4085-9111-7f495bc41228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:34.622299 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc2b2b-d13c-4085-9111-7f495bc41228 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:34.622572 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc2b2b-d13c-4085-9111-7f495bc41228 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:34.622837 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc2b2b-d13c-4085-9111-7f495bc41228 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:34.637291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc2b2b-d13c-4085-9111-7f495bc41228 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:34.642291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-d6bc2b2b-d13c-4085-9111-7f495bc41228 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:24:34.642291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:34.642291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:34.642291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:34.642291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:34.642291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:34.642291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:34.642291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:34.642291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:34.642291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:34.642291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:34.642291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:34.642291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:34.642291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:34.642291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:34.642291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:34.644609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:34.644609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:34.644609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:34.644609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:34.644609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:34.644609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:34.644609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:34.644609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:34.644609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:34.644609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:34.644609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:34.644609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:34.644609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:34.644609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:34.644609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:34.644609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:34.646701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:34.646701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:34.646701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:34.646701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:34.646701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:34.646701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:34.646701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:34.646701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:34.646701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:34.646701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:34.646701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 22:24:34.646701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:34.646701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 241/479] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:24:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:24:34.652526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352f3ce-56ad-4b58-9596-cfea9a583181 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:34.756635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:34.767227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352f3ce-56ad-4b58-9596-cfea9a583181 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:34.802202 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49590ae-35a8-4530-b76e-0eedf50ea8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:34.802809 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49590ae-35a8-4530-b76e-0eedf50ea8af None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:34.803087 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49590ae-35a8-4530-b76e-0eedf50ea8af None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:34.803389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49590ae-35a8-4530-b76e-0eedf50ea8af None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:34.804054 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49590ae-35a8-4530-b76e-0eedf50ea8af None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:34.804368 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49590ae-35a8-4530-b76e-0eedf50ea8af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:34.806209 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49590ae-35a8-4530-b76e-0eedf50ea8af None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:34.815078 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 239/480] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:35.634252 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303d5ea-c3fb-46d3-9bfb-05637ee6855e None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:35.635336 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303d5ea-c3fb-46d3-9bfb-05637ee6855e None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:35.635819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303d5ea-c3fb-46d3-9bfb-05637ee6855e None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:35.637670 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 242/481] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:35.643528 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288ff42-1857-4ecb-94ae-575708a53880 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:35.644436 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288ff42-1857-4ecb-94ae-575708a53880 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:35.644923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288ff42-1857-4ecb-94ae-575708a53880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:35.673360 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-9288ff42-1857-4ecb-94ae-575708a53880 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:35.675001 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-9288ff42-1857-4ecb-94ae-575708a53880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:35.767573 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 240/482] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:35.878031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1e30a-8c7d-4320-9483-f6bc8a21bce8 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:35.878648 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1e30a-8c7d-4320-9483-f6bc8a21bce8 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:35.879119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1e30a-8c7d-4320-9483-f6bc8a21bce8 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:35.880949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 243/483] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:35.887188 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b94762-50c6-4bf4-96ee-0d5f1ec61724 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:35.887818 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b94762-50c6-4bf4-96ee-0d5f1ec61724 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:35.888337 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b94762-50c6-4bf4-96ee-0d5f1ec61724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:35.917593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-13b94762-50c6-4bf4-96ee-0d5f1ec61724 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:35.919270 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-13b94762-50c6-4bf4-96ee-0d5f1ec61724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:36.002620 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 241/484] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:36.145553 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01520957-246b-40f4-a165-9ada4cb51798 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:36.145877 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01520957-246b-40f4-a165-9ada4cb51798 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:36.146048 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01520957-246b-40f4-a165-9ada4cb51798 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:36.147597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 244/485] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:36.153305 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff243b4f-914a-4692-8418-7fb10ba83123 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:36.153593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff243b4f-914a-4692-8418-7fb10ba83123 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:36.153869 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff243b4f-914a-4692-8418-7fb10ba83123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:36.159259 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01520957-246b-40f4-a165-9ada4cb51798 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:36.198680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-ff243b4f-914a-4692-8418-7fb10ba83123 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:36.201140 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ff243b4f-914a-4692-8418-7fb10ba83123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:36.224248 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:36.333868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01520957-246b-40f4-a165-9ada4cb51798 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:36.359148 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 242/486] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:36.364756 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff243b4f-914a-4692-8418-7fb10ba83123 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:36.387939 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fde489-9698-4f97-977d-c9950f7656ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:36.388704 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fde489-9698-4f97-977d-c9950f7656ff None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:36.389542 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fde489-9698-4f97-977d-c9950f7656ff None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:36.389818 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fde489-9698-4f97-977d-c9950f7656ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:36.414466 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fde489-9698-4f97-977d-c9950f7656ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:36.426305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-92fde489-9698-4f97-977d-c9950f7656ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:36.426305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:36.426305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:36.426305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:36.426305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:36.426305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:36.426305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:36.426305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:36.426305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:36.426305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:36.426305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:36.426305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:36.426305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:36.426305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:36.426305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:36.426305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:36.427213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:36.427213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:36.427213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:36.427213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:36.427213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:36.427213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:36.427213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:36.427213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:36.427213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:36.427213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:36.427213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:36.427213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:36.427213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:36.427213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:36.427213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:36.427213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:36.428021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:36.428021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:36.428021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:36.428021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:36.428021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:36.428021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:36.428021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:36.428021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:36.428021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:36.428021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:36.428021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:36.428021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:36.428021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 245/487] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:24:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:24:36.434250 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fde489-9698-4f97-977d-c9950f7656ff None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:36.489163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:36.550622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff243b4f-914a-4692-8418-7fb10ba83123 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:36.592606 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cb239-8454-4050-bdbd-96cb025d8156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:36.593253 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cb239-8454-4050-bdbd-96cb025d8156 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:36.593585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cb239-8454-4050-bdbd-96cb025d8156 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:36.593859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cb239-8454-4050-bdbd-96cb025d8156 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:36.616554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cb239-8454-4050-bdbd-96cb025d8156 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:36.625310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-cc1cb239-8454-4050-bdbd-96cb025d8156 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:36.625310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:36.625310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:36.625310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:36.625310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:36.625310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:36.625310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:36.625310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:36.625310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:36.625310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:36.625310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:36.625310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:36.625310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:36.625310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:36.625310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:36.625310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:36.626220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:36.626220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:36.626220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:36.626220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:36.626220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:36.626220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:36.626220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:36.626220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:36.626220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:36.626220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:36.626220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:36.626220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:36.626220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:36.626220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:36.626220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:36.626220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:36.627599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:36.627599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:36.627599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:36.627599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:36.627599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:36.627599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:36.627599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:36.627599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:36.627599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:36.627599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:36.627599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:36.627599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:36.627599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 243/488] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:24:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:36.634980 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cb239-8454-4050-bdbd-96cb025d8156 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:36.640932 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:36.664007 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fde489-9698-4f97-977d-c9950f7656ff None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:36.705164 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da69037-c15f-4bcd-9e12-47fc7b092c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:36.706203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da69037-c15f-4bcd-9e12-47fc7b092c89 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:36.706685 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da69037-c15f-4bcd-9e12-47fc7b092c89 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:36.707256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da69037-c15f-4bcd-9e12-47fc7b092c89 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:36.708167 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da69037-c15f-4bcd-9e12-47fc7b092c89 None admin] RBAC: Adding query filter params (name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:36.708688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da69037-c15f-4bcd-9e12-47fc7b092c89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:36.710419 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da69037-c15f-4bcd-9e12-47fc7b092c89 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:36.721344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 246/489] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:24:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:36.722711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da69037-c15f-4bcd-9e12-47fc7b092c89 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:36.776277 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cb239-8454-4050-bdbd-96cb025d8156 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:36.827573 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a567310-8ab4-4df3-88e8-135f14605ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:36.828959 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:36.830102 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a567310-8ab4-4df3-88e8-135f14605ec2 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:36.830512 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a567310-8ab4-4df3-88e8-135f14605ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:36.831277 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a567310-8ab4-4df3-88e8-135f14605ec2 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:36.832077 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a567310-8ab4-4df3-88e8-135f14605ec2 None admin] RBAC: Adding query filter params (name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:36.832504 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a567310-8ab4-4df3-88e8-135f14605ec2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:36.834793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a567310-8ab4-4df3-88e8-135f14605ec2 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:36.844518 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da69037-c15f-4bcd-9e12-47fc7b092c89 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:36.851600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 244/490] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:24:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:36.851600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a567310-8ab4-4df3-88e8-135f14605ec2 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:36.888530 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ac4df-03db-446e-9c23-526f2ea92801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:36.889488 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ac4df-03db-446e-9c23-526f2ea92801 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:36.889913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ac4df-03db-446e-9c23-526f2ea92801 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:36.890393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ac4df-03db-446e-9c23-526f2ea92801 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:36.903829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ac4df-03db-446e-9c23-526f2ea92801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:36.912121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-412ac4df-03db-446e-9c23-526f2ea92801 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:36.912121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:36.912121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:36.912121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:36.912121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:36.912121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:36.912121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:36.912121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:36.912121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:36.912121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:36.912121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:36.912121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:36.912121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:36.912121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:36.912121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:36.912121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:36.913081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:36.913081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:36.913081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:36.913081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:36.913081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:36.913081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:36.913081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:36.913081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:36.913081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:36.913081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:36.913081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:36.913081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:36.913081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:36.913081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:36.913081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:36.913081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:36.914029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:36.914029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:36.914029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:36.914029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:36.914029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:36.914029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:36.914029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:36.914029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:36.914029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:36.914029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:36.914029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:36.914029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:36.917134 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 247/491] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:24:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:36.918196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ac4df-03db-446e-9c23-526f2ea92801 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:36.939168 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:36.994433 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a567310-8ab4-4df3-88e8-135f14605ec2 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:37.048493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62d3b4-707f-41a1-91da-c9fc67535051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:37.049310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62d3b4-707f-41a1-91da-c9fc67535051 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:37.049928 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62d3b4-707f-41a1-91da-c9fc67535051 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:37.050302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62d3b4-707f-41a1-91da-c9fc67535051 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:37.053894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ac4df-03db-446e-9c23-526f2ea92801 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:37.063677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62d3b4-707f-41a1-91da-c9fc67535051 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:37.065502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62d3b4-707f-41a1-91da-c9fc67535051 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:37.081841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-8b62d3b4-707f-41a1-91da-c9fc67535051 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:37.081841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:37.081841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:37.081841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:37.081841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:37.081841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:37.081841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:37.081841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:37.081841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:37.081841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:37.081841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:37.081841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:37.081841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:37.081841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:37.081841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:37.081841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:37.082802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:37.082802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:37.082802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:37.082802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:37.082802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:37.082802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:37.082802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:37.082802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:37.082802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:37.082802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:37.082802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:37.082802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:37.082802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:37.082802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:37.082802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:37.082802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:37.082802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:37.083650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:37.083650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:37.083650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:37.085536 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 245/492] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:37.086595 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62d3b4-707f-41a1-91da-c9fc67535051 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:37.102717 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb04117-fdb7-4744-88ce-2c51423815d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:37.103855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb04117-fdb7-4744-88ce-2c51423815d0 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:37.104247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb04117-fdb7-4744-88ce-2c51423815d0 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:37.104526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb04117-fdb7-4744-88ce-2c51423815d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:37.121381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb04117-fdb7-4744-88ce-2c51423815d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:37.123584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb04117-fdb7-4744-88ce-2c51423815d0 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:37.134548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:37.141548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-9fb04117-fdb7-4744-88ce-2c51423815d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:37.141548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:37.141548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:37.141548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:37.141548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:37.141548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:37.141548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:37.141548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:37.141548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:37.141548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:37.141548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:37.141548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:37.141548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:37.141548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:37.141548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:37.141548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:37.142574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:37.142574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:37.142574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:37.142574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:37.142574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:37.142574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:37.142574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:37.142574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:37.142574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:37.142574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:37.142574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:37.142574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:37.142574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:37.142574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:37.142574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:37.142574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:37.142574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:37.143448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:37.143448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:37.143448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:37.145271 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 248/493] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:37.150868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb04117-fdb7-4744-88ce-2c51423815d0 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:37.221004 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62d3b4-707f-41a1-91da-c9fc67535051 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:37.244381 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:37.265614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eae6c5-c1d0-48e8-897e-b364b4e7a589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:37.266143 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eae6c5-c1d0-48e8-897e-b364b4e7a589 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:37.266427 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eae6c5-c1d0-48e8-897e-b364b4e7a589 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:37.266702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eae6c5-c1d0-48e8-897e-b364b4e7a589 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:37.267376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eae6c5-c1d0-48e8-897e-b364b4e7a589 None admin] RBAC: Adding query filter params (name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:37.267680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eae6c5-c1d0-48e8-897e-b364b4e7a589 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:37.268853 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb04117-fdb7-4744-88ce-2c51423815d0 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:37.269199 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eae6c5-c1d0-48e8-897e-b364b4e7a589 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:37.281252 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 246/494] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:24:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:37.286612 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eae6c5-c1d0-48e8-897e-b364b4e7a589 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:37.305478 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acc234-2dee-4661-aeb5-1509ae7c8cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:37.306728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acc234-2dee-4661-aeb5-1509ae7c8cb4 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:37.307219 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acc234-2dee-4661-aeb5-1509ae7c8cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:37.307507 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acc234-2dee-4661-aeb5-1509ae7c8cb4 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:37.308162 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acc234-2dee-4661-aeb5-1509ae7c8cb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:37.308478 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acc234-2dee-4661-aeb5-1509ae7c8cb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:37.309944 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acc234-2dee-4661-aeb5-1509ae7c8cb4 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:37.322809 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 249/495] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:37.324106 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32566f6-0552-46c2-8168-8f383333b174 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:37.324426 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32566f6-0552-46c2-8168-8f383333b174 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:37.324696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32566f6-0552-46c2-8168-8f383333b174 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:37.326062 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 250/496] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:37.330820 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32566f6-0552-46c2-8168-8f383333b174 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:37.398597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:37.401017 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eae6c5-c1d0-48e8-897e-b364b4e7a589 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:37.437032 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3038fc3-7fcb-473e-a42a-11345beec980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:37.437643 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3038fc3-7fcb-473e-a42a-11345beec980 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:37.437917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3038fc3-7fcb-473e-a42a-11345beec980 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:37.438185 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3038fc3-7fcb-473e-a42a-11345beec980 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:37.438823 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3038fc3-7fcb-473e-a42a-11345beec980 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:37.439125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3038fc3-7fcb-473e-a42a-11345beec980 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:37.440511 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3038fc3-7fcb-473e-a42a-11345beec980 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:37.441368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32566f6-0552-46c2-8168-8f383333b174 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:37.452589 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 247/497] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:37.453459 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3038fc3-7fcb-473e-a42a-11345beec980 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:37.478634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c5479-e0c4-4a47-b3e5-c7ac8a93a2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:37.479356 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c5479-e0c4-4a47-b3e5-c7ac8a93a2a2 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:37.479630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c5479-e0c4-4a47-b3e5-c7ac8a93a2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:37.479900 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c5479-e0c4-4a47-b3e5-c7ac8a93a2a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:37.490607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c5479-e0c4-4a47-b3e5-c7ac8a93a2a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:37.492197 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c5479-e0c4-4a47-b3e5-c7ac8a93a2a2 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:37.495245 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:37.508171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-494c5479-e0c4-4a47-b3e5-c7ac8a93a2a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:37.508171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:37.508171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:37.508171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:37.508171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:37.508171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:37.508171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:37.508171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:37.508171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:37.508171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:37.508171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:37.508171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:37.508171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:37.508171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:37.508171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:37.508171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:37.509501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:37.509501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:37.509501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:37.509501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:37.509501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:37.509501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:37.509501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:37.509501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:37.509501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:37.509501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:37.509501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:37.509501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:37.509501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:37.509501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:37.509501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:37.509501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:37.509501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:37.510548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:37.510548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:37.510548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:37.510548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 251/498] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:37.510770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8adb35-98a9-475e-b78a-0d1b34eb1d3c None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:37.510950 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8adb35-98a9-475e-b78a-0d1b34eb1d3c None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:37.511321 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8adb35-98a9-475e-b78a-0d1b34eb1d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:37.531245 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-6c8adb35-98a9-475e-b78a-0d1b34eb1d3c None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:37.532408 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6c8adb35-98a9-475e-b78a-0d1b34eb1d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:37.572195 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3038fc3-7fcb-473e-a42a-11345beec980 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:37.602149 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 252/499] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:37.604498 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36a247-ed0f-4e4f-aea7-02eed755ce86 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:37.604796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36a247-ed0f-4e4f-aea7-02eed755ce86 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:37.605064 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36a247-ed0f-4e4f-aea7-02eed755ce86 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:37.608291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 253/500] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:37.608291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36a247-ed0f-4e4f-aea7-02eed755ce86 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:37.614722 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764d2dc-5dd3-49a9-93b7-12bfc658a43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:37.615378 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764d2dc-5dd3-49a9-93b7-12bfc658a43d None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:37.615656 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764d2dc-5dd3-49a9-93b7-12bfc658a43d None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:37.615964 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764d2dc-5dd3-49a9-93b7-12bfc658a43d None admin] PATH_INFO: `/v3/users/nova` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:37.626080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764d2dc-5dd3-49a9-93b7-12bfc658a43d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:37.627539 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764d2dc-5dd3-49a9-93b7-12bfc658a43d None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:37.641842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-5764d2dc-5dd3-49a9-93b7-12bfc658a43d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:24:37.641842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:37.641842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:37.641842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:37.641842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:37.641842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:37.641842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:37.641842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:37.641842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:37.641842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:37.641842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:37.641842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:37.641842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:24:37.641842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:24:37.641842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:24:37.641842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:24:37.642816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:37.642816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:37.642816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:24:37.642816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:37.642816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:24:37.642816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:37.642816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:37.642816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:37.642816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:37.642816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:37.642816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:37.642816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:37.642816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:37.642816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:37.642816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:37.642816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:37.642816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:37.642816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:37.643698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:37.643698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:37.643698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:24:37.643698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:24:37.643698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:24:37.643698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:24:37.643698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:24:37.643698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:24:37.643698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:24:37.643698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:37.643698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 248/501] 10.222.0.59 () {56 vars in 1093 bytes} [Mon Aug 29 22:24:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:37.645181 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764d2dc-5dd3-49a9-93b7-12bfc658a43d None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:37.669378 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:37.729638 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36a247-ed0f-4e4f-aea7-02eed755ce86 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:37.756541 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:37.760893 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764d2dc-5dd3-49a9-93b7-12bfc658a43d None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:37.769051 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41018173-f121-4717-bec9-15d07a42c412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:37.769678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41018173-f121-4717-bec9-15d07a42c412 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:37.769956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41018173-f121-4717-bec9-15d07a42c412 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:37.770226 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41018173-f121-4717-bec9-15d07a42c412 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:37.778714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41018173-f121-4717-bec9-15d07a42c412 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:37.780193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41018173-f121-4717-bec9-15d07a42c412 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:37.795298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-41018173-f121-4717-bec9-15d07a42c412 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 22:24:37.795298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:37.795298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:37.795298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:37.795298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:37.795298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:37.795298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:37.795298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:37.795298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:37.795298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:37.795298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:37.795298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:37.795298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:24:37.795298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:24:37.795298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:24:37.795298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:24:37.796417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:37.796417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:37.796417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:24:37.796417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:37.796417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:24:37.796417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:37.796417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:37.796417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:37.796417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:37.796417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:37.796417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:37.796417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:37.796417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:37.796417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:37.796417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:37.796417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:37.796417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:37.796417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:37.797496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:37.797496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:37.797496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:24:37.797496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:24:37.797496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:24:37.797496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:24:37.797496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:24:37.797496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:24:37.797496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 22:24:37.797496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:37.801110 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 254/502] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:37.801879 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f088bb3-afff-4d81-bf19-0aacee50c1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:37.802339 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a2122-1235-4421-b9c5-dc107fa881e5 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:37.802546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a2122-1235-4421-b9c5-dc107fa881e5 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:37.802821 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a2122-1235-4421-b9c5-dc107fa881e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:37.803374 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f088bb3-afff-4d81-bf19-0aacee50c1c6 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:37.803784 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f088bb3-afff-4d81-bf19-0aacee50c1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:37.804213 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f088bb3-afff-4d81-bf19-0aacee50c1c6 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:37.805005 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f088bb3-afff-4d81-bf19-0aacee50c1c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:37.805436 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f088bb3-afff-4d81-bf19-0aacee50c1c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:37.806994 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f088bb3-afff-4d81-bf19-0aacee50c1c6 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:37.825063 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 249/503] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:37.825063 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f088bb3-afff-4d81-bf19-0aacee50c1c6 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:37.825303 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-f38a2122-1235-4421-b9c5-dc107fa881e5 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:37.825303 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f38a2122-1235-4421-b9c5-dc107fa881e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:37.905893 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 255/504] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:37.906745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a2122-1235-4421-b9c5-dc107fa881e5 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:37.942871 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:37.959618 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f088bb3-afff-4d81-bf19-0aacee50c1c6 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:38.010130 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba66ef-0e57-41d3-8f64-5ad2c2081fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:38.011118 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba66ef-0e57-41d3-8f64-5ad2c2081fee None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:38.011589 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba66ef-0e57-41d3-8f64-5ad2c2081fee None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:38.012165 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba66ef-0e57-41d3-8f64-5ad2c2081fee None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:38.013675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba66ef-0e57-41d3-8f64-5ad2c2081fee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:38.014378 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba66ef-0e57-41d3-8f64-5ad2c2081fee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:38.016289 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba66ef-0e57-41d3-8f64-5ad2c2081fee None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:38.026585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a2122-1235-4421-b9c5-dc107fa881e5 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:38.064091 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 250/505] 10.222.0.59 () {56 vars in 1133 bytes} [Mon Aug 29 22:24:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:38.065248 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba66ef-0e57-41d3-8f64-5ad2c2081fee None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:38.083126 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3534e7-1a1b-4280-868e-be5b23568d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:38.083765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3534e7-1a1b-4280-868e-be5b23568d31 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:38.084006 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3534e7-1a1b-4280-868e-be5b23568d31 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:38.084298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3534e7-1a1b-4280-868e-be5b23568d31 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:38.085024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3534e7-1a1b-4280-868e-be5b23568d31 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:38.085323 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3534e7-1a1b-4280-868e-be5b23568d31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:38.085691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:38.086757 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3534e7-1a1b-4280-868e-be5b23568d31 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:38.126976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 256/506] 10.222.0.59 () {56 vars in 1137 bytes} [Mon Aug 29 22:24:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:38.127817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3534e7-1a1b-4280-868e-be5b23568d31 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:38.177450 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba66ef-0e57-41d3-8f64-5ad2c2081fee None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:38.214794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6765b-9281-4842-ba7a-b3eb975594a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:38.215476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6765b-9281-4842-ba7a-b3eb975594a8 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:38.215812 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6765b-9281-4842-ba7a-b3eb975594a8 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:38.216153 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6765b-9281-4842-ba7a-b3eb975594a8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:38.222852 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:38.223817 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6765b-9281-4842-ba7a-b3eb975594a8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:38.234264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6765b-9281-4842-ba7a-b3eb975594a8 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:38.237576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-f1e6765b-9281-4842-ba7a-b3eb975594a8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 22:24:38.237576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:38.237576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:38.237576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:38.237576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:38.237576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:38.237576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:38.237576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:38.237576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:38.237576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:38.237576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:38.237576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:38.237576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:24:38.237576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:24:38.237576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:24:38.237576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:24:38.238678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:38.238678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:38.238678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:24:38.238678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:38.238678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:24:38.238678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:38.238678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:38.238678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:38.238678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:38.238678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:38.238678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:38.238678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:38.238678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:38.238678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:38.238678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:38.238678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:38.238678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:38.238678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:38.239629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:38.239629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:38.239629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:24:38.239629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:24:38.239629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:24:38.239629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:24:38.239629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:24:38.239629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:24:38.239629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 22:24:38.239629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:38.240339 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3534e7-1a1b-4280-868e-be5b23568d31 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:38.240421 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 251/507] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:38.240421 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6765b-9281-4842-ba7a-b3eb975594a8 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:38.289203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2584a00-1153-4b15-a13b-b6a1190dd0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:38.289847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2584a00-1153-4b15-a13b-b6a1190dd0d6 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:38.290146 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2584a00-1153-4b15-a13b-b6a1190dd0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:38.290419 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2584a00-1153-4b15-a13b-b6a1190dd0d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:38.302088 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2584a00-1153-4b15-a13b-b6a1190dd0d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:38.307027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-d2584a00-1153-4b15-a13b-b6a1190dd0d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:38.307027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:38.307027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:38.307027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:38.307027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:38.307027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:38.307027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:38.307027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:38.307027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:38.307027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:38.307027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:38.307027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:38.307027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:38.307027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:38.307027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:38.307027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:38.307983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:38.307983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:38.307983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:38.307983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:38.307983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:38.307983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:38.307983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:38.307983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:38.307983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:38.307983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:38.307983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:38.307983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:38.307983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:38.307983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:38.307983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:38.307983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:38.308925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:38.308925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:38.308925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:38.308925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:38.308925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:38.308925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:38.308925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:38.308925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:38.308925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:38.308925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:38.308925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:38.308925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:38.308925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 257/508] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:24:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:38.309572 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2584a00-1153-4b15-a13b-b6a1190dd0d6 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:38.327808 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:38.360292 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6765b-9281-4842-ba7a-b3eb975594a8 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:38.404427 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93d606-bbab-4b42-9ffc-4ae98fea13a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:38.405352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93d606-bbab-4b42-9ffc-4ae98fea13a8 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:38.405791 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93d606-bbab-4b42-9ffc-4ae98fea13a8 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:38.406226 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93d606-bbab-4b42-9ffc-4ae98fea13a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:38.438740 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2584a00-1153-4b15-a13b-b6a1190dd0d6 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:38.473089 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b29b3f-6e53-48aa-a5f1-7fda352c054a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:38.473717 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b29b3f-6e53-48aa-a5f1-7fda352c054a None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:38.473991 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b29b3f-6e53-48aa-a5f1-7fda352c054a None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:38.474261 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b29b3f-6e53-48aa-a5f1-7fda352c054a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:38.478466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93d606-bbab-4b42-9ffc-4ae98fea13a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:38.479912 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93d606-bbab-4b42-9ffc-4ae98fea13a8 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:38.495091 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:38.544906 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b29b3f-6e53-48aa-a5f1-7fda352c054a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:38.546377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b29b3f-6e53-48aa-a5f1-7fda352c054a None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:38.556661 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:38.583057 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 252/509] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:38.583937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93d606-bbab-4b42-9ffc-4ae98fea13a8 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:38.654414 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 258/510] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:38.655357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b29b3f-6e53-48aa-a5f1-7fda352c054a None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:38.705535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93d606-bbab-4b42-9ffc-4ae98fea13a8 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:38.709873 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:38.741004 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbd51a-370b-4094-9f7d-75f9e510f503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:38.741675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbd51a-370b-4094-9f7d-75f9e510f503 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:38.741900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbd51a-370b-4094-9f7d-75f9e510f503 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:38.742159 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbd51a-370b-4094-9f7d-75f9e510f503 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:38.752594 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbd51a-370b-4094-9f7d-75f9e510f503 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:38.754194 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbd51a-370b-4094-9f7d-75f9e510f503 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:38.762209 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b29b3f-6e53-48aa-a5f1-7fda352c054a None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:38.769206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-11fbd51a-370b-4094-9f7d-75f9e510f503 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:38.769206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:38.769206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:38.769206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:38.769206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:38.769206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:38.769206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:38.769206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:38.769206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:38.769206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:38.769206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:38.769206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:38.769206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:38.769206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:38.769206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:38.769206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:38.770488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:38.770488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:38.770488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:38.770488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:38.770488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:38.770488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:38.770488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:38.770488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:38.770488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:38.770488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:38.770488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:38.770488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:38.770488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:38.770488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:38.770488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:38.770488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:38.770488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:38.771643 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:38.771643 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:38.771643 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:38.771643 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 253/511] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:38.771643 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbd51a-370b-4094-9f7d-75f9e510f503 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:38.800225 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c272d70-2325-467f-a108-475292074a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:38.800823 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c272d70-2325-467f-a108-475292074a87 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:38.801096 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c272d70-2325-467f-a108-475292074a87 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:38.801360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c272d70-2325-467f-a108-475292074a87 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:38.802034 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:38.802461 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c272d70-2325-467f-a108-475292074a87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:38.802461 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c272d70-2325-467f-a108-475292074a87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:38.803757 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c272d70-2325-467f-a108-475292074a87 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:38.820711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 259/512] 10.222.0.59 () {56 vars in 1137 bytes} [Mon Aug 29 22:24:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:38.821942 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c272d70-2325-467f-a108-475292074a87 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:38.878666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbd51a-370b-4094-9f7d-75f9e510f503 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:38.912954 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3507bd-ea27-45a9-bdd3-9b258326257b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:38.913579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3507bd-ea27-45a9-bdd3-9b258326257b None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:38.913849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3507bd-ea27-45a9-bdd3-9b258326257b None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:38.914144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3507bd-ea27-45a9-bdd3-9b258326257b None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:38.917311 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3507bd-ea27-45a9-bdd3-9b258326257b None admin] RBAC: Adding query filter params (name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:38.917311 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3507bd-ea27-45a9-bdd3-9b258326257b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:38.917311 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3507bd-ea27-45a9-bdd3-9b258326257b None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:38.923124 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c272d70-2325-467f-a108-475292074a87 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:38.927956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 254/513] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:24:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:38.929074 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3507bd-ea27-45a9-bdd3-9b258326257b None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:38.931472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:38.957051 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae3347-a787-4795-87fe-545045867c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:38.957685 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae3347-a787-4795-87fe-545045867c42 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:38.957966 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae3347-a787-4795-87fe-545045867c42 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:38.958233 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae3347-a787-4795-87fe-545045867c42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:38.969510 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae3347-a787-4795-87fe-545045867c42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:38.970965 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae3347-a787-4795-87fe-545045867c42 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:38.985581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-03ae3347-a787-4795-87fe-545045867c42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:38.985581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:38.985581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:38.985581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:38.985581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:38.985581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:38.985581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:38.985581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:38.985581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:38.985581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:38.985581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:38.985581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:38.985581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:38.985581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:38.985581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:38.985581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:38.986511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:38.986511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:38.986511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:38.986511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:38.986511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:38.986511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:38.986511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:38.986511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:38.986511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:38.986511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:38.986511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:38.986511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:38.986511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:38.986511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:38.986511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:38.986511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:38.986511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:38.987388 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:38.987388 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:38.987388 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:38.987388 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 260/514] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:38.987600 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae3347-a787-4795-87fe-545045867c42 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:39.028065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:39.037185 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3507bd-ea27-45a9-bdd3-9b258326257b None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:39.073282 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20378207-5384-4e72-baf6-4f34a757d02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:39.073937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20378207-5384-4e72-baf6-4f34a757d02b None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:39.074214 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20378207-5384-4e72-baf6-4f34a757d02b None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:39.074486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20378207-5384-4e72-baf6-4f34a757d02b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:39.085391 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20378207-5384-4e72-baf6-4f34a757d02b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:39.086901 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20378207-5384-4e72-baf6-4f34a757d02b None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:39.092383 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae3347-a787-4795-87fe-545045867c42 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:39.101746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-20378207-5384-4e72-baf6-4f34a757d02b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:39.101746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:39.101746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:39.101746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:39.101746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:39.101746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:39.101746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:39.101746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:39.101746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:39.101746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:39.101746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:39.101746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:39.101746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:39.101746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:39.101746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:39.101746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:39.102738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:39.102738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:39.102738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:39.102738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:39.102738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:39.102738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:39.102738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:39.102738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:39.102738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:39.102738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:39.102738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:39.102738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:39.102738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:39.102738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:39.102738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:39.102738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:39.102738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:39.103666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:39.103666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:39.103666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:39.103666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 255/515] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:39.104085 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20378207-5384-4e72-baf6-4f34a757d02b None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:39.129622 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0202e1d4-1903-4729-abf0-1342d95c92b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:39.130144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202e1d4-1903-4729-abf0-1342d95c92b8 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:39.130420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202e1d4-1903-4729-abf0-1342d95c92b8 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:39.130688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202e1d4-1903-4729-abf0-1342d95c92b8 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:39.131326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0202e1d4-1903-4729-abf0-1342d95c92b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:39.131622 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0202e1d4-1903-4729-abf0-1342d95c92b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:39.133060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0202e1d4-1903-4729-abf0-1342d95c92b8 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:39.145212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 261/516] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:39.146009 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0202e1d4-1903-4729-abf0-1342d95c92b8 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:39.168872 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:39.214560 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20378207-5384-4e72-baf6-4f34a757d02b None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:39.248440 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0202e1d4-1903-4729-abf0-1342d95c92b8 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:39.248621 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e265e-cbf1-44c7-821b-e6f633d0da8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:39.249139 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e265e-cbf1-44c7-821b-e6f633d0da8b None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:39.249412 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e265e-cbf1-44c7-821b-e6f633d0da8b None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:39.249713 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e265e-cbf1-44c7-821b-e6f633d0da8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:39.263623 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:39.282959 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334ed3c-c3ec-43fd-b127-2a4045a11afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:39.283573 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334ed3c-c3ec-43fd-b127-2a4045a11afe None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:39.283845 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334ed3c-c3ec-43fd-b127-2a4045a11afe None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:39.284114 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334ed3c-c3ec-43fd-b127-2a4045a11afe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:39.294619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334ed3c-c3ec-43fd-b127-2a4045a11afe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:39.296053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334ed3c-c3ec-43fd-b127-2a4045a11afe None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:39.310326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-2334ed3c-c3ec-43fd-b127-2a4045a11afe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:39.310326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:39.310326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:39.310326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:39.310326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:39.310326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:39.310326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:39.310326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:39.310326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:39.310326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:39.310326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:39.310326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:39.310326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:39.310326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:39.310326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:39.310326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:39.311437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:39.311437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:39.311437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:39.311437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:39.311437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:39.311437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:39.311437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:39.311437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:39.311437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:39.311437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:39.311437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:39.311437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:39.311437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:39.311437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:39.311437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:39.311437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:39.311437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:39.312481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:39.312481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:39.312481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:39.312481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 262/517] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:39.312705 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334ed3c-c3ec-43fd-b127-2a4045a11afe None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:39.324811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e265e-cbf1-44c7-821b-e6f633d0da8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:39.326260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e265e-cbf1-44c7-821b-e6f633d0da8b None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:39.390480 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:39.421386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334ed3c-c3ec-43fd-b127-2a4045a11afe None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:39.431105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 256/518] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:39.431105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e265e-cbf1-44c7-821b-e6f633d0da8b None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:39.456912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517ab68-b6da-416d-bf3e-03cb9486d700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:39.457511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517ab68-b6da-416d-bf3e-03cb9486d700 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:39.457798 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517ab68-b6da-416d-bf3e-03cb9486d700 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:39.458050 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517ab68-b6da-416d-bf3e-03cb9486d700 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:39.458678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517ab68-b6da-416d-bf3e-03cb9486d700 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:39.458968 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517ab68-b6da-416d-bf3e-03cb9486d700 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:39.460362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517ab68-b6da-416d-bf3e-03cb9486d700 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:39.472643 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:39.473107 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 263/519] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:39.473922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517ab68-b6da-416d-bf3e-03cb9486d700 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:39.531769 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e265e-cbf1-44c7-821b-e6f633d0da8b None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:39.567403 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297b47a-de59-4c92-975f-d4257488e2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:39.568005 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297b47a-de59-4c92-975f-d4257488e2a4 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:39.568307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297b47a-de59-4c92-975f-d4257488e2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:39.568595 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297b47a-de59-4c92-975f-d4257488e2a4 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:39.569218 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4297b47a-de59-4c92-975f-d4257488e2a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:39.569518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4297b47a-de59-4c92-975f-d4257488e2a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:39.570927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4297b47a-de59-4c92-975f-d4257488e2a4 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:39.574805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517ab68-b6da-416d-bf3e-03cb9486d700 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:39.584588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 257/520] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:39.585298 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297b47a-de59-4c92-975f-d4257488e2a4 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:39.612081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:39.617054 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c8ba2-a2e2-413c-8f48-0b4b385c60fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:39.617659 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c8ba2-a2e2-413c-8f48-0b4b385c60fd None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:39.617949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c8ba2-a2e2-413c-8f48-0b4b385c60fd None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:39.618233 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c8ba2-a2e2-413c-8f48-0b4b385c60fd None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:39.620796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c8ba2-a2e2-413c-8f48-0b4b385c60fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:39.620796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c8ba2-a2e2-413c-8f48-0b4b385c60fd None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:39.689324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297b47a-de59-4c92-975f-d4257488e2a4 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:39.690727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:39.699113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 264/521] 10.222.0.59 () {60 vars in 1132 bytes} [Mon Aug 29 22:24:39 2022] POST /identity/v3/users => generated 247 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:24:39.699944 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c8ba2-a2e2-413c-8f48-0b4b385c60fd None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:39.734334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9268c-131f-44d3-8d19-3f9db0041861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:39.735218 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9268c-131f-44d3-8d19-3f9db0041861 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:39.735707 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9268c-131f-44d3-8d19-3f9db0041861 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:39.736202 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9268c-131f-44d3-8d19-3f9db0041861 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:39.737150 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9268c-131f-44d3-8d19-3f9db0041861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:39.737594 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9268c-131f-44d3-8d19-3f9db0041861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:39.739124 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9268c-131f-44d3-8d19-3f9db0041861 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:39.755382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 258/522] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:39.756698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9268c-131f-44d3-8d19-3f9db0041861 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:39.826233 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c8ba2-a2e2-413c-8f48-0b4b385c60fd None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:39.867248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483906e-a7f5-4a78-8b4c-0254fbb76470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:39.867877 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483906e-a7f5-4a78-8b4c-0254fbb76470 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:39.868236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483906e-a7f5-4a78-8b4c-0254fbb76470 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:39.868616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483906e-a7f5-4a78-8b4c-0254fbb76470 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:39.877074 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9268c-131f-44d3-8d19-3f9db0041861 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:39.884477 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483906e-a7f5-4a78-8b4c-0254fbb76470 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:39.885967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483906e-a7f5-4a78-8b4c-0254fbb76470 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:39.903526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-0483906e-a7f5-4a78-8b4c-0254fbb76470 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:39.903526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:39.903526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:39.903526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:39.903526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:39.903526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:39.903526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:39.903526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:39.903526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:39.903526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:39.903526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:39.903526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:39.903526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:39.903526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:39.903526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:39.903526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:39.904672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:39.904672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:39.904672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:39.904672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:39.904672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:39.904672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:39.904672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:39.904672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:39.904672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:39.904672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:39.904672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:39.904672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:39.904672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:39.904672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:39.904672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:39.904672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:39.904672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:39.905512 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:39.905512 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:39.905512 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:39.905512 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 265/523] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:39.905512 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483906e-a7f5-4a78-8b4c-0254fbb76470 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:39.923334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af8a39-4232-4b4c-99a2-408ee757d57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:39.923334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af8a39-4232-4b4c-99a2-408ee757d57b None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:39.923556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af8a39-4232-4b4c-99a2-408ee757d57b None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:39.923762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af8a39-4232-4b4c-99a2-408ee757d57b None admin] PATH_INFO: `/v3/projects/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:39.931445 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af8a39-4232-4b4c-99a2-408ee757d57b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:39.933012 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af8a39-4232-4b4c-99a2-408ee757d57b None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:39.945556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-d3af8a39-4232-4b4c-99a2-408ee757d57b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:39.945556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:39.945556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:39.945556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:39.945556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:39.945556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:39.945556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:39.945556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:39.945556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:39.945556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:39.945556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:39.945556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:39.945556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:24:39.945556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:24:39.945556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:24:39.947045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:24:39.947045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:39.947045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:39.947045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:39.947045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:39.947045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:39.947045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:39.947045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:39.947045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:39.947045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:39.947045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:39.947045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:39.947045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:39.947045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:39.947045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:39.947045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:39.947045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:39.947888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:24:39.947888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:24:39.947888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:24:39.947888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:24:39.947888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:24:39.947888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:24:39.947888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:39.947888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:39.947888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 259/524] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:39.948454 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af8a39-4232-4b4c-99a2-408ee757d57b None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:39.953809 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:39.976679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:40.017211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483906e-a7f5-4a78-8b4c-0254fbb76470 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:40.052801 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a4fb7-e3fa-4843-82cd-d500ccd54a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:40.053426 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a4fb7-e3fa-4843-82cd-d500ccd54a9e None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:40.053701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a4fb7-e3fa-4843-82cd-d500ccd54a9e None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:40.053954 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a4fb7-e3fa-4843-82cd-d500ccd54a9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:40.061319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af8a39-4232-4b4c-99a2-408ee757d57b None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:40.061617 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a4fb7-e3fa-4843-82cd-d500ccd54a9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:40.063063 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a4fb7-e3fa-4843-82cd-d500ccd54a9e None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:40.076790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-0e7a4fb7-e3fa-4843-82cd-d500ccd54a9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:40.076790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:40.076790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:40.076790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:40.076790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:40.076790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:40.076790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:40.076790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:40.076790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:40.076790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:40.076790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:40.076790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:40.076790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:24:40.076790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:24:40.076790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:24:40.077728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:24:40.077728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:40.077728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:40.077728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:40.077728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:40.077728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:40.077728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:40.077728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:40.077728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:40.077728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:40.077728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:40.077728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:40.077728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:40.077728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:40.077728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:40.077728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:40.077728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:40.078661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:24:40.078661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:24:40.078661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:24:40.078661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:24:40.078661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:24:40.078661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:24:40.078661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:40.078661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:40.078661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 266/525] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:40.079127 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a4fb7-e3fa-4843-82cd-d500ccd54a9e None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:40.107353 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b48618-2c1b-4453-ae3c-537173dd4cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:40.108045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b48618-2c1b-4453-ae3c-537173dd4cdc None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:40.108349 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b48618-2c1b-4453-ae3c-537173dd4cdc None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:40.108647 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b48618-2c1b-4453-ae3c-537173dd4cdc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:40.117077 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b48618-2c1b-4453-ae3c-537173dd4cdc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:40.119554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b48618-2c1b-4453-ae3c-537173dd4cdc None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:40.132557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-55b48618-2c1b-4453-ae3c-537173dd4cdc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 22:24:40.132557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:40.132557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:40.132557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:40.132557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:40.132557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:40.132557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:40.132557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:40.132557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:40.132557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:40.132557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:40.132557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:40.132557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:24:40.132557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:24:40.132557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:24:40.132557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:24:40.133484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:40.133484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:40.133484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:24:40.133484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:40.133484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:24:40.133484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:40.133484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:40.133484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:40.133484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:40.133484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:40.133484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:40.133484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:40.133484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:40.133484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:40.133484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:40.133484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:40.133484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:40.133484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:40.135217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:40.135217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:40.135217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:24:40.135217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:24:40.135217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:24:40.135217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:24:40.135217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:24:40.135217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:24:40.135217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 22:24:40.135217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:40.135217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 260/526] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:24:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:40.135779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b48618-2c1b-4453-ae3c-537173dd4cdc None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:40.192794 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:40.204532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a4fb7-e3fa-4843-82cd-d500ccd54a9e None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:40.222380 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:40.235857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93f826-845a-4e45-aba5-d9604fd4b40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:40.236641 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93f826-845a-4e45-aba5-d9604fd4b40b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:40.237206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93f826-845a-4e45-aba5-d9604fd4b40b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:40.237495 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93f826-845a-4e45-aba5-d9604fd4b40b None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:40.238398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93f826-845a-4e45-aba5-d9604fd4b40b None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:40.238732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93f826-845a-4e45-aba5-d9604fd4b40b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:40.239728 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b48618-2c1b-4453-ae3c-537173dd4cdc None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:40.240121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93f826-845a-4e45-aba5-d9604fd4b40b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:40.256184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 267/527] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:40.257058 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93f826-845a-4e45-aba5-d9604fd4b40b None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:40.282758 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966dad9e-af2c-4a10-be6f-1a6655c1bf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:40.283430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966dad9e-af2c-4a10-be6f-1a6655c1bf04 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:40.283722 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966dad9e-af2c-4a10-be6f-1a6655c1bf04 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:40.284007 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966dad9e-af2c-4a10-be6f-1a6655c1bf04 None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:40.284752 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966dad9e-af2c-4a10-be6f-1a6655c1bf04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:40.285060 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966dad9e-af2c-4a10-be6f-1a6655c1bf04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:40.286488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966dad9e-af2c-4a10-be6f-1a6655c1bf04 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:40.306474 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 261/528] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:24:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:40.307205 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966dad9e-af2c-4a10-be6f-1a6655c1bf04 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:40.374302 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93f826-845a-4e45-aba5-d9604fd4b40b None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:40.411415 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f17d4f-9c1c-4f92-93c2-20a3d6026c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:40.411680 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f17d4f-9c1c-4f92-93c2-20a3d6026c1a None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:40.411934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f17d4f-9c1c-4f92-93c2-20a3d6026c1a None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:40.412235 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f17d4f-9c1c-4f92-93c2-20a3d6026c1a None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:40.412927 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f17d4f-9c1c-4f92-93c2-20a3d6026c1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:40.413228 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f17d4f-9c1c-4f92-93c2-20a3d6026c1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:40.414649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f17d4f-9c1c-4f92-93c2-20a3d6026c1a None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:40.417711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966dad9e-af2c-4a10-be6f-1a6655c1bf04 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:40.438838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 268/529] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:24:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:40.439555 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f17d4f-9c1c-4f92-93c2-20a3d6026c1a None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:40.455560 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97533e-5790-4fe6-bf33-7ac0db8bcbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:40.456207 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97533e-5790-4fe6-bf33-7ac0db8bcbc0 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:40.456481 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97533e-5790-4fe6-bf33-7ac0db8bcbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:40.456749 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97533e-5790-4fe6-bf33-7ac0db8bcbc0 None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:40.457470 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97533e-5790-4fe6-bf33-7ac0db8bcbc0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:40.457765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97533e-5790-4fe6-bf33-7ac0db8bcbc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:40.459170 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97533e-5790-4fe6-bf33-7ac0db8bcbc0 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:40.463460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:40.474406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:40.484218 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 262/530] 10.222.0.59 () {56 vars in 1139 bytes} [Mon Aug 29 22:24:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:40.484218 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97533e-5790-4fe6-bf33-7ac0db8bcbc0 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:40.545148 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f17d4f-9c1c-4f92-93c2-20a3d6026c1a None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:40.581708 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4b816-3091-46c3-880a-6c26a9d60410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:40.582355 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4b816-3091-46c3-880a-6c26a9d60410 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:40.582684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4b816-3091-46c3-880a-6c26a9d60410 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:40.582950 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4b816-3091-46c3-880a-6c26a9d60410 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:40.592211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f4b816-3091-46c3-880a-6c26a9d60410 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:40.593663 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f4b816-3091-46c3-880a-6c26a9d60410 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:40.594407 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97533e-5790-4fe6-bf33-7ac0db8bcbc0 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:40.605870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-21f4b816-3091-46c3-880a-6c26a9d60410 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:40.605870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:40.605870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:40.605870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:40.605870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:40.605870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:40.605870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:40.605870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:40.605870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:40.605870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:40.605870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:40.605870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:40.605870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:24:40.605870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:24:40.605870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:24:40.606804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:24:40.606804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:40.606804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:40.606804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:40.606804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:40.606804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:40.606804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:40.606804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:40.606804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:40.606804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:40.606804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:40.606804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:40.606804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:40.606804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:40.606804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:40.606804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:40.606804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:40.607647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:24:40.607647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:24:40.607647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:24:40.607647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:24:40.607647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:24:40.607647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:24:40.607647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:40.607647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:40.607647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 269/531] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:40.608110 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4b816-3091-46c3-880a-6c26a9d60410 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:40.640025 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6c20a-6199-4c6a-b344-2f26c5cbb206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:40.640667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6c20a-6199-4c6a-b344-2f26c5cbb206 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:40.640952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6c20a-6199-4c6a-b344-2f26c5cbb206 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:40.641223 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6c20a-6199-4c6a-b344-2f26c5cbb206 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:40.641958 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6c20a-6199-4c6a-b344-2f26c5cbb206 None admin] RBAC: Adding query filter params (role.id=f91580fb2e5840bcb4f4054732e2e796, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=7f21105679f245a19ea4850e6c010dbf) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:40.642263 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6c20a-6199-4c6a-b344-2f26c5cbb206 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:40.643685 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6c20a-6199-4c6a-b344-2f26c5cbb206 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:40.649811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 263/532] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:24:40 2022] GET /identity/v3/role_assignments?user.id=7f21105679f245a19ea4850e6c010dbf&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=f91580fb2e5840bcb4f4054732e2e796 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:40.651145 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6c20a-6199-4c6a-b344-2f26c5cbb206 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:40.732605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:40.746831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:40.760130 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4b816-3091-46c3-880a-6c26a9d60410 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:40.806291 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6c20a-6199-4c6a-b344-2f26c5cbb206 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:40.811739 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d45ffc-b0be-44f4-951a-026e12f4c7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:40.812401 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d45ffc-b0be-44f4-951a-026e12f4c7ad None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:40.812684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d45ffc-b0be-44f4-951a-026e12f4c7ad None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:40.812954 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d45ffc-b0be-44f4-951a-026e12f4c7ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:40.813691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d45ffc-b0be-44f4-951a-026e12f4c7ad None admin] RBAC: Adding query filter params (role.id=f91580fb2e5840bcb4f4054732e2e796, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=92af56c889e640e7815c1641c05a9af3) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:40.813990 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d45ffc-b0be-44f4-951a-026e12f4c7ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:40.815429 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d45ffc-b0be-44f4-951a-026e12f4c7ad None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:40.822195 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 270/533] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:24:40 2022] GET /identity/v3/role_assignments?user.id=92af56c889e640e7815c1641c05a9af3&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=f91580fb2e5840bcb4f4054732e2e796 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:40.823051 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d45ffc-b0be-44f4-951a-026e12f4c7ad None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:40.893920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15141eb0-a195-4eab-bb35-9a8e7dc7ab3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:40.894545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15141eb0-a195-4eab-bb35-9a8e7dc7ab3c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:40.894828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15141eb0-a195-4eab-bb35-9a8e7dc7ab3c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:40.895097 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15141eb0-a195-4eab-bb35-9a8e7dc7ab3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:41.002455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15141eb0-a195-4eab-bb35-9a8e7dc7ab3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:41.004003 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15141eb0-a195-4eab-bb35-9a8e7dc7ab3c None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:41.026782 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d45ffc-b0be-44f4-951a-026e12f4c7ad None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:41.084654 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:41.113414 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002d7cd-4705-4a0e-8316-f5821857bc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:41.114182 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002d7cd-4705-4a0e-8316-f5821857bc93 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:41.114555 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002d7cd-4705-4a0e-8316-f5821857bc93 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:41.114917 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002d7cd-4705-4a0e-8316-f5821857bc93 None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:41.115706 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002d7cd-4705-4a0e-8316-f5821857bc93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:41.118036 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002d7cd-4705-4a0e-8316-f5821857bc93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:41.119906 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002d7cd-4705-4a0e-8316-f5821857bc93 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:41.147808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 271/534] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:24:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:41.154672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002d7cd-4705-4a0e-8316-f5821857bc93 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:41.156987 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 264/535] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:41.161768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:41.166069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15141eb0-a195-4eab-bb35-9a8e7dc7ab3c None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:41.343283 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002d7cd-4705-4a0e-8316-f5821857bc93 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:41.347413 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15141eb0-a195-4eab-bb35-9a8e7dc7ab3c None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:41.386701 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee2d67-d6f9-4277-984e-65a423cefe00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:41.387369 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee2d67-d6f9-4277-984e-65a423cefe00 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:41.387640 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee2d67-d6f9-4277-984e-65a423cefe00 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:41.387906 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee2d67-d6f9-4277-984e-65a423cefe00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:41.399154 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee2d67-d6f9-4277-984e-65a423cefe00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:41.401325 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee2d67-d6f9-4277-984e-65a423cefe00 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:41.417493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-51ee2d67-d6f9-4277-984e-65a423cefe00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:41.417493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:41.417493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:41.417493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:41.417493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:41.417493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:41.417493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:41.417493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:41.417493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:41.417493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:41.417493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:41.417493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:41.417493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:41.417493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:41.417493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:41.417493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:41.418624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:41.418624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:41.418624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:41.418624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:41.418624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:41.418624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:41.418624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:41.418624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:41.418624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:41.418624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:41.418624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:41.418624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:41.418624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:41.418624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:41.418624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:41.418624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:41.418624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:41.419665 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:41.419665 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:41.419665 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:41.419665 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 265/536] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:41.424735 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137cb8f-21d7-4edf-ba7e-848dccb81798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:41.424886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee2d67-d6f9-4277-984e-65a423cefe00 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:41.425638 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137cb8f-21d7-4edf-ba7e-848dccb81798 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:41.426059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137cb8f-21d7-4edf-ba7e-848dccb81798 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:41.426488 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137cb8f-21d7-4edf-ba7e-848dccb81798 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:41.427375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137cb8f-21d7-4edf-ba7e-848dccb81798 None admin] RBAC: Adding query filter params (role.id=f91580fb2e5840bcb4f4054732e2e796, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=4db08aa2e4ef4478a3909a876754814c) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:41.427831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137cb8f-21d7-4edf-ba7e-848dccb81798 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:41.429446 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137cb8f-21d7-4edf-ba7e-848dccb81798 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:41.440785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 272/537] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:24:41 2022] GET /identity/v3/role_assignments?user.id=4db08aa2e4ef4478a3909a876754814c&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=f91580fb2e5840bcb4f4054732e2e796 => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:41.445405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:41.541527 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee2d67-d6f9-4277-984e-65a423cefe00 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:41.581883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad7d05-2406-4198-9fd9-05a244a9a77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:41.582638 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad7d05-2406-4198-9fd9-05a244a9a77e None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:41.582985 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad7d05-2406-4198-9fd9-05a244a9a77e None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:41.583319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad7d05-2406-4198-9fd9-05a244a9a77e None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:41.584050 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad7d05-2406-4198-9fd9-05a244a9a77e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:41.584518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad7d05-2406-4198-9fd9-05a244a9a77e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:41.585911 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad7d05-2406-4198-9fd9-05a244a9a77e None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:41.600029 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 266/538] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:41.606703 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137cb8f-21d7-4edf-ba7e-848dccb81798 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:41.705751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:41.744358 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137cb8f-21d7-4edf-ba7e-848dccb81798 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:41.786268 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c033b88-963d-4957-9997-12099435d619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:41.787013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c033b88-963d-4957-9997-12099435d619 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:41.787288 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c033b88-963d-4957-9997-12099435d619 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:41.787598 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c033b88-963d-4957-9997-12099435d619 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:41.795484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c033b88-963d-4957-9997-12099435d619 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:41.797094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c033b88-963d-4957-9997-12099435d619 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:41.809879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-1c033b88-963d-4957-9997-12099435d619 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:41.809879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:41.809879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:41.809879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:41.809879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:41.809879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:41.809879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:41.809879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:41.809879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:41.809879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:41.809879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:41.809879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:41.809879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:24:41.809879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:24:41.809879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:24:41.810759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:24:41.810759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:41.810759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:41.810759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:41.810759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:41.810759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:41.810759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:41.810759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:41.810759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:41.810759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:41.810759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:41.810759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:41.810759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:41.810759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:41.810759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:41.810759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:41.810759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:41.811595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:24:41.811595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:24:41.811595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:24:41.811595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:24:41.811595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:24:41.811595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:24:41.811595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:41.811595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:41.811595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 273/539] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:41.817336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad7d05-2406-4198-9fd9-05a244a9a77e None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:41.890366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088ab6e-6bcd-4457-b2c4-8bc459703642 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:41.890741 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088ab6e-6bcd-4457-b2c4-8bc459703642 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:41.890902 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088ab6e-6bcd-4457-b2c4-8bc459703642 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:41.892367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 274/540] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:24:41.897541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6655c22-39e9-4285-9952-93b4e8fa30c2 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:41.898082 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6655c22-39e9-4285-9952-93b4e8fa30c2 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:41.898567 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6655c22-39e9-4285-9952-93b4e8fa30c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:41.930785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-a6655c22-39e9-4285-9952-93b4e8fa30c2 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:41.937113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a6655c22-39e9-4285-9952-93b4e8fa30c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:41.942053 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:41.990726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad7d05-2406-4198-9fd9-05a244a9a77e None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:42.062647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 275/541] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:24:42.063652 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81a7dc-f099-45a8-8640-a6e3901e6cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:42.064303 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81a7dc-f099-45a8-8640-a6e3901e6cd1 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:42.064584 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81a7dc-f099-45a8-8640-a6e3901e6cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:42.064856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81a7dc-f099-45a8-8640-a6e3901e6cd1 None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:42.065633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81a7dc-f099-45a8-8640-a6e3901e6cd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:42.065938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81a7dc-f099-45a8-8640-a6e3901e6cd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:42.067375 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81a7dc-f099-45a8-8640-a6e3901e6cd1 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:42.088110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 267/542] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:24:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:42.094130 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6655c22-39e9-4285-9952-93b4e8fa30c2 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:42.175655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:42.235016 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6655c22-39e9-4285-9952-93b4e8fa30c2 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:42.288462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ed8e6-1389-43b7-98d9-a856f33a1b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:42.289074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ed8e6-1389-43b7-98d9-a856f33a1b31 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:42.289351 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ed8e6-1389-43b7-98d9-a856f33a1b31 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:42.289623 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ed8e6-1389-43b7-98d9-a856f33a1b31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:42.290927 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ed8e6-1389-43b7-98d9-a856f33a1b31 None admin] RBAC: Adding query filter params (role.id=f91580fb2e5840bcb4f4054732e2e796, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=cb802d01e71b486cb40c6388430d674d) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:42.291353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ed8e6-1389-43b7-98d9-a856f33a1b31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:42.292901 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ed8e6-1389-43b7-98d9-a856f33a1b31 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:42.303054 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 276/543] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:24:42 2022] GET /identity/v3/role_assignments?user.id=cb802d01e71b486cb40c6388430d674d&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=f91580fb2e5840bcb4f4054732e2e796 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:42.367405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81a7dc-f099-45a8-8640-a6e3901e6cd1 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:42.509818 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81a7dc-f099-45a8-8640-a6e3901e6cd1 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:42.557307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114b314-b30a-4a80-86f9-b446c4546704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:42.558315 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114b314-b30a-4a80-86f9-b446c4546704 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:42.558796 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114b314-b30a-4a80-86f9-b446c4546704 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:42.559286 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114b314-b30a-4a80-86f9-b446c4546704 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:42.569877 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:42.579788 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114b314-b30a-4a80-86f9-b446c4546704 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:42.584943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-b114b314-b30a-4a80-86f9-b446c4546704 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:42.584943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:42.584943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:42.584943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:42.584943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:42.584943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:42.584943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:42.584943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:42.584943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:42.584943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:42.584943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:42.584943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:42.584943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:42.584943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:42.584943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:42.584943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:42.595309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:42.595309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:42.595309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:42.595309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:42.595309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:42.595309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:42.595309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:42.595309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:42.595309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:42.595309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:42.595309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:42.595309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:42.595309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:42.595309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:42.595309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:42.595309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:42.596089 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:42.596089 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:42.596089 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:42.596089 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:42.596089 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:42.596089 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:42.596089 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:42.596089 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:42.596089 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:42.596089 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:42.596089 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:42.596089 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:42.596089 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 268/544] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:24:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:42.596779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ed8e6-1389-43b7-98d9-a856f33a1b31 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:42.716334 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ed8e6-1389-43b7-98d9-a856f33a1b31 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:42.747119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:42.771915 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56d0bd-1452-4a56-9585-b1c97282c802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:42.771915 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56d0bd-1452-4a56-9585-b1c97282c802 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:42.771915 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56d0bd-1452-4a56-9585-b1c97282c802 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:42.771915 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56d0bd-1452-4a56-9585-b1c97282c802 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:42.772319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56d0bd-1452-4a56-9585-b1c97282c802 None admin] RBAC: Adding query filter params (name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:42.772319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56d0bd-1452-4a56-9585-b1c97282c802 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:42.772319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56d0bd-1452-4a56-9585-b1c97282c802 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:42.776347 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 277/545] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:24:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:42.782427 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114b314-b30a-4a80-86f9-b446c4546704 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:42.931264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114b314-b30a-4a80-86f9-b446c4546704 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:42.980757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92227bbe-6499-477b-ac8f-31300f895ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:42.982217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92227bbe-6499-477b-ac8f-31300f895ae6 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:42.982698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92227bbe-6499-477b-ac8f-31300f895ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:42.983167 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92227bbe-6499-477b-ac8f-31300f895ae6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:43.002959 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92227bbe-6499-477b-ac8f-31300f895ae6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:43.004851 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92227bbe-6499-477b-ac8f-31300f895ae6 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:43.024301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-92227bbe-6499-477b-ac8f-31300f895ae6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:43.024301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:43.024301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:43.024301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:43.024301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:43.024301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:43.024301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:43.024301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:43.024301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:43.024301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:43.024301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:43.024301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:43.024301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:43.024301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:43.024301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:43.024301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:43.025389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:43.025389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:43.025389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:43.025389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:43.025389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:43.025389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:43.025389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:43.025389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:43.025389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:43.025389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:43.025389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:43.025389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:43.025389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:43.025389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:43.025389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:43.025389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:43.025389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:43.026234 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:43.026234 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:43.026234 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:43.028481 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 269/546] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:24:43.037264 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56d0bd-1452-4a56-9585-b1c97282c802 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:43.237752 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56d0bd-1452-4a56-9585-b1c97282c802 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:43.240051 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3b29c-7700-4cbb-9b22-174f830b2b06 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:43.240610 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3b29c-7700-4cbb-9b22-174f830b2b06 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:43.241097 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3b29c-7700-4cbb-9b22-174f830b2b06 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:43.242902 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 270/547] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:43.249565 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08dd8e8-48bb-4441-8d7f-66249e693c60 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:43.249895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08dd8e8-48bb-4441-8d7f-66249e693c60 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:43.250263 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08dd8e8-48bb-4441-8d7f-66249e693c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:43.286482 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-a08dd8e8-48bb-4441-8d7f-66249e693c60 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:43.288692 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a08dd8e8-48bb-4441-8d7f-66249e693c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:43.296102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c9d3d-58a0-4f8f-8a5c-42a49eb31b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:43.297057 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c9d3d-58a0-4f8f-8a5c-42a49eb31b5c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:43.297594 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c9d3d-58a0-4f8f-8a5c-42a49eb31b5c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:43.298042 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c9d3d-58a0-4f8f-8a5c-42a49eb31b5c None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:43.298943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c9d3d-58a0-4f8f-8a5c-42a49eb31b5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:43.300152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c9d3d-58a0-4f8f-8a5c-42a49eb31b5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:43.301831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c9d3d-58a0-4f8f-8a5c-42a49eb31b5c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:43.322420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 278/548] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:43.328153 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c9d3d-58a0-4f8f-8a5c-42a49eb31b5c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:43.377643 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:43.386595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:43.417798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 271/549] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:43.419298 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda4113-d9a4-42f9-84bc-0e4c040d754c None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:43.419632 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda4113-d9a4-42f9-84bc-0e4c040d754c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:43.419903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda4113-d9a4-42f9-84bc-0e4c040d754c None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:43.421557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 272/550] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:43.426933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9ddd8-c43b-4cda-9fbf-079e962759ed None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:43.427211 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9ddd8-c43b-4cda-9fbf-079e962759ed None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:43.427510 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9ddd8-c43b-4cda-9fbf-079e962759ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:43.449858 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c9d3d-58a0-4f8f-8a5c-42a49eb31b5c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:43.465282 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-b5a9ddd8-c43b-4cda-9fbf-079e962759ed None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:43.466490 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b5a9ddd8-c43b-4cda-9fbf-079e962759ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:43.485696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5ea8d-ff94-4e56-a16e-3187eb236577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:43.486450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5ea8d-ff94-4e56-a16e-3187eb236577 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:43.486735 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5ea8d-ff94-4e56-a16e-3187eb236577 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:43.487006 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5ea8d-ff94-4e56-a16e-3187eb236577 None admin] PATH_INFO: `/v3/users/swift` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:43.495456 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5ea8d-ff94-4e56-a16e-3187eb236577 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:43.496966 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5ea8d-ff94-4e56-a16e-3187eb236577 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:43.510590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-bab5ea8d-ff94-4e56-a16e-3187eb236577 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:24:43.510590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:43.510590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:43.510590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:43.510590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:43.510590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:43.510590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:43.510590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:43.510590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:43.510590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:43.510590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:43.510590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:43.510590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:24:43.510590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:24:43.510590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:24:43.510590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:24:43.511585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:43.511585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:43.511585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:24:43.511585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:43.511585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:24:43.511585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:43.511585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:43.511585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:43.511585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:43.511585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:43.511585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:43.511585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:43.511585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:43.511585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:43.511585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:43.511585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:43.511585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:43.511585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:43.512609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:43.512609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:43.512609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:24:43.512609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:24:43.512609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:24:43.512609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:24:43.512609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:24:43.512609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:24:43.512609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:24:43.512609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:43.512609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 279/551] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:24:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:43.517721 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5ea8d-ff94-4e56-a16e-3187eb236577 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:43.627896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5ea8d-ff94-4e56-a16e-3187eb236577 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:43.634458 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:43.669903 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebc4a1-09b8-470f-9c1d-6675acb00477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:43.670519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebc4a1-09b8-470f-9c1d-6675acb00477 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:43.670800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebc4a1-09b8-470f-9c1d-6675acb00477 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:43.671141 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebc4a1-09b8-470f-9c1d-6675acb00477 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:43.671884 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebc4a1-09b8-470f-9c1d-6675acb00477 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:43.672214 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebc4a1-09b8-470f-9c1d-6675acb00477 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:43.673680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 273/552] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:43.673801 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebc4a1-09b8-470f-9c1d-6675acb00477 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:43.701394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 280/553] 10.222.0.59 () {56 vars in 1135 bytes} [Mon Aug 29 22:24:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:43.706704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9ddd8-c43b-4cda-9fbf-079e962759ed None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:43.830312 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebc4a1-09b8-470f-9c1d-6675acb00477 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:43.875363 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9ddd8-c43b-4cda-9fbf-079e962759ed None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:43.886495 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:43.918060 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd420132-666b-4b92-af5d-dd93cd21f441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:43.918688 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd420132-666b-4b92-af5d-dd93cd21f441 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:43.919033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd420132-666b-4b92-af5d-dd93cd21f441 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:43.919442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd420132-666b-4b92-af5d-dd93cd21f441 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:43.986808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebc4a1-09b8-470f-9c1d-6675acb00477 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:44.023053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba57198-c1fb-46b8-ba50-4b9fdfa877fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:44.024083 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba57198-c1fb-46b8-ba50-4b9fdfa877fd None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:44.024676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba57198-c1fb-46b8-ba50-4b9fdfa877fd None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:44.025165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba57198-c1fb-46b8-ba50-4b9fdfa877fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:44.035961 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba57198-c1fb-46b8-ba50-4b9fdfa877fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:44.041275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-9ba57198-c1fb-46b8-ba50-4b9fdfa877fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:44.041275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:44.041275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:44.041275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:44.041275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:44.041275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:44.041275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:44.041275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:44.041275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:44.041275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:44.041275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:44.041275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:44.041275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:44.041275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:44.041275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:44.041275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:44.042292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:44.042292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:44.042292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:44.042292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:44.042292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:44.042292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:44.042292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:44.042292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:44.042292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:44.042292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:44.042292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:44.042292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:44.042292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:44.042292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:44.042292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:44.042292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:44.043112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:44.043112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:44.043112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:44.043112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:44.043112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:44.043112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:44.043112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:44.043112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:44.043112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:44.043112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:44.043112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:44.043112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:44.045739 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 281/554] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:24:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:24:44.051623 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba57198-c1fb-46b8-ba50-4b9fdfa877fd None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:44.064922 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd420132-666b-4b92-af5d-dd93cd21f441 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:44.067036 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd420132-666b-4b92-af5d-dd93cd21f441 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:44.095646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:44.174165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba57198-c1fb-46b8-ba50-4b9fdfa877fd None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:44.184048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 274/555] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:44.189314 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd420132-666b-4b92-af5d-dd93cd21f441 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:44.201027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:44.213103 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624a302-54cd-4293-9f4f-ae67be29652d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:44.213868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624a302-54cd-4293-9f4f-ae67be29652d None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:44.214153 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624a302-54cd-4293-9f4f-ae67be29652d None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:44.214421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624a302-54cd-4293-9f4f-ae67be29652d None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:44.215081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624a302-54cd-4293-9f4f-ae67be29652d None admin] RBAC: Adding query filter params (name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:44.215631 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624a302-54cd-4293-9f4f-ae67be29652d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:44.216942 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624a302-54cd-4293-9f4f-ae67be29652d None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:44.232270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 282/556] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:24:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:44.232270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80b047-fdae-4957-b8ac-6425e0998f3a None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:44.232270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80b047-fdae-4957-b8ac-6425e0998f3a None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:44.232270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80b047-fdae-4957-b8ac-6425e0998f3a None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:44.232270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 283/557] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:44.232270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80b047-fdae-4957-b8ac-6425e0998f3a None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:44.296067 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd420132-666b-4b92-af5d-dd93cd21f441 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:44.326702 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:44.335931 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efc732-ca3c-488d-b6d4-a49a68bcaaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:44.337776 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efc732-ca3c-488d-b6d4-a49a68bcaaaa None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:44.338480 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efc732-ca3c-488d-b6d4-a49a68bcaaaa None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:44.339001 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efc732-ca3c-488d-b6d4-a49a68bcaaaa None admin] PATH_INFO: `/v3/roles/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:44.344767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80b047-fdae-4957-b8ac-6425e0998f3a None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:44.351088 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efc732-ca3c-488d-b6d4-a49a68bcaaaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:44.356578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-04efc732-ca3c-488d-b6d4-a49a68bcaaaa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:44.356578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:44.356578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:44.356578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:44.356578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:44.356578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:44.356578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:44.356578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:44.356578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:44.356578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:44.356578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:44.356578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:44.356578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:44.356578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:44.356578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:44.356578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:44.357537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:44.357537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:44.357537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:44.357537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:44.357537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:44.357537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:44.357537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:44.357537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:44.357537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:44.357537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:44.357537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:44.357537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:44.357537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:44.357537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:44.357537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:44.357537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:44.358409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:44.358409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:44.358409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:44.358409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:44.358409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:44.358409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:44.358409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:44.358409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:44.358409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:44.358409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:44.358409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:44.358409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:44.358409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 275/558] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:24:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:44.359836 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efc732-ca3c-488d-b6d4-a49a68bcaaaa None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:44.388381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdadbdc6-5554-45c7-8eff-65cb16ad71f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:44.389013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdadbdc6-5554-45c7-8eff-65cb16ad71f7 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:44.389286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdadbdc6-5554-45c7-8eff-65cb16ad71f7 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:44.389551 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdadbdc6-5554-45c7-8eff-65cb16ad71f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:44.401263 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdadbdc6-5554-45c7-8eff-65cb16ad71f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:44.402775 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdadbdc6-5554-45c7-8eff-65cb16ad71f7 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:44.417838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-cdadbdc6-5554-45c7-8eff-65cb16ad71f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:44.417838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:44.417838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:44.417838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:44.417838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:44.417838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:44.417838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:44.417838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:44.417838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:44.417838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:44.417838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:44.417838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:44.417838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:44.417838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:44.417838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:44.417838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:44.419052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:44.419052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:44.419052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:44.419052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:44.419052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:44.419052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:44.419052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:44.419052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:44.419052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:44.419052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:44.419052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:44.419052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:44.419052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:44.419052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:44.419052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:44.419052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:44.419052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:44.420078 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:44.420078 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:44.420078 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:44.420078 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 284/559] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:44.420796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc26fc3-d7e0-4e7f-9eb8-8d2b7b558160 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:44.420796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc26fc3-d7e0-4e7f-9eb8-8d2b7b558160 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:44.420956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc26fc3-d7e0-4e7f-9eb8-8d2b7b558160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:44.439068 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-dcc26fc3-d7e0-4e7f-9eb8-8d2b7b558160 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:44.440184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-dcc26fc3-d7e0-4e7f-9eb8-8d2b7b558160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:44.442444 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:44.479764 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efc732-ca3c-488d-b6d4-a49a68bcaaaa None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:44.512913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 285/560] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:44.518195 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb6c9d-2005-4d97-9beb-02d36942c2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:44.518830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb6c9d-2005-4d97-9beb-02d36942c2aa None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:44.519112 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb6c9d-2005-4d97-9beb-02d36942c2aa None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:44.519374 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb6c9d-2005-4d97-9beb-02d36942c2aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:44.525171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc26fc3-d7e0-4e7f-9eb8-8d2b7b558160 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:44.532719 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb6c9d-2005-4d97-9beb-02d36942c2aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:44.534170 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb6c9d-2005-4d97-9beb-02d36942c2aa None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:44.549999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-b3eb6c9d-2005-4d97-9beb-02d36942c2aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:44.549999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:44.549999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:44.549999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:44.549999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:44.549999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:44.549999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:44.549999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:44.549999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:44.549999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:44.549999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:44.549999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:44.549999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:44.549999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:44.549999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:44.549999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:44.551092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:44.551092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:44.551092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:44.551092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:44.551092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:44.551092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:44.551092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:44.551092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:44.551092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:44.551092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:44.551092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:44.551092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:44.551092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:44.551092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:44.551092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:44.551092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:44.551092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:44.551951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:44.551951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:44.551951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:44.551951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 276/561] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:44.551951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb6c9d-2005-4d97-9beb-02d36942c2aa None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:44.616774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:44.667462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc26fc3-d7e0-4e7f-9eb8-8d2b7b558160 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:44.676444 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb6c9d-2005-4d97-9beb-02d36942c2aa None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:44.700322 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:44.717521 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df4b1ef-cf42-4cd6-8047-882f7cb75146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:44.718139 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4b1ef-cf42-4cd6-8047-882f7cb75146 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:44.718419 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4b1ef-cf42-4cd6-8047-882f7cb75146 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:44.718691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4b1ef-cf42-4cd6-8047-882f7cb75146 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:44.719397 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df4b1ef-cf42-4cd6-8047-882f7cb75146 None admin] RBAC: Adding query filter params (name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:44.719700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df4b1ef-cf42-4cd6-8047-882f7cb75146 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:44.721181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df4b1ef-cf42-4cd6-8047-882f7cb75146 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:44.721877 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c084564-24ab-4b8b-a8dc-e2c21ddf5312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:44.723093 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c084564-24ab-4b8b-a8dc-e2c21ddf5312 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:44.724067 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c084564-24ab-4b8b-a8dc-e2c21ddf5312 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:44.724590 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c084564-24ab-4b8b-a8dc-e2c21ddf5312 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:44.725706 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c084564-24ab-4b8b-a8dc-e2c21ddf5312 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:44.726232 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c084564-24ab-4b8b-a8dc-e2c21ddf5312 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:44.728766 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c084564-24ab-4b8b-a8dc-e2c21ddf5312 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:44.730922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 286/562] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:24:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:44.731394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df4b1ef-cf42-4cd6-8047-882f7cb75146 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:44.748417 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 277/563] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:44.749837 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c084564-24ab-4b8b-a8dc-e2c21ddf5312 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:44.846782 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df4b1ef-cf42-4cd6-8047-882f7cb75146 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:44.871075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c084564-24ab-4b8b-a8dc-e2c21ddf5312 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:44.874219 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:44.882165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac7f39-8960-4476-9b1c-1be91eec4c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:44.882772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac7f39-8960-4476-9b1c-1be91eec4c8e None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:44.883048 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac7f39-8960-4476-9b1c-1be91eec4c8e None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:44.883384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac7f39-8960-4476-9b1c-1be91eec4c8e None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:44.885724 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac7f39-8960-4476-9b1c-1be91eec4c8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:44.885724 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac7f39-8960-4476-9b1c-1be91eec4c8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:44.886057 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac7f39-8960-4476-9b1c-1be91eec4c8e None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:44.897834 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 287/564] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:44.898712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac7f39-8960-4476-9b1c-1be91eec4c8e None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:44.908780 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff844bb-679c-4cd3-ae4f-b761b5e73679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:44.909412 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff844bb-679c-4cd3-ae4f-b761b5e73679 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:44.909687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff844bb-679c-4cd3-ae4f-b761b5e73679 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:44.909952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff844bb-679c-4cd3-ae4f-b761b5e73679 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:44.921001 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff844bb-679c-4cd3-ae4f-b761b5e73679 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:44.922584 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff844bb-679c-4cd3-ae4f-b761b5e73679 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:44.938448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-fff844bb-679c-4cd3-ae4f-b761b5e73679 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:44.938448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:44.938448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:44.938448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:44.938448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:44.938448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:44.938448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:44.938448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:44.938448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:44.938448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:44.938448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:44.938448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:44.938448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:44.938448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:44.938448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:44.938448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:44.939591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:44.939591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:44.939591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:44.939591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:44.939591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:44.939591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:44.939591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:44.939591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:44.939591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:44.939591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:44.939591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:44.939591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:44.939591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:44.939591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:44.939591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:44.939591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:44.939591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:44.940523 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:44.940523 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:44.940523 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:44.940523 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 278/565] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:44.940762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff844bb-679c-4cd3-ae4f-b761b5e73679 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:44.971848 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:45.003176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac7f39-8960-4476-9b1c-1be91eec4c8e None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:45.037235 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899dfcef-013a-4f76-a69f-89fdc3fc7a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:45.037853 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dfcef-013a-4f76-a69f-89fdc3fc7a68 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:45.038123 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dfcef-013a-4f76-a69f-89fdc3fc7a68 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:45.038395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dfcef-013a-4f76-a69f-89fdc3fc7a68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:45.042210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff844bb-679c-4cd3-ae4f-b761b5e73679 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:45.045927 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899dfcef-013a-4f76-a69f-89fdc3fc7a68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:45.047366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899dfcef-013a-4f76-a69f-89fdc3fc7a68 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:45.059829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-899dfcef-013a-4f76-a69f-89fdc3fc7a68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:45.059829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:45.059829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:45.059829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:45.059829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:45.059829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:45.059829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:45.059829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:45.059829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:45.059829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:45.059829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:45.059829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:45.059829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:24:45.059829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:24:45.059829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:24:45.060742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:24:45.060742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:45.060742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:45.060742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:45.060742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:45.060742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:45.060742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:45.060742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:45.060742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:45.060742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:45.060742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:45.060742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:45.060742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:45.060742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:45.060742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:45.060742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:45.060742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:45.061609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:24:45.061609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:24:45.061609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:24:45.061609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:24:45.061609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:24:45.061609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:24:45.061609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:45.061609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:45.061609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 288/566] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:45.062506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81b9a1-1064-4ed3-a2cb-1bf2a2e7341e None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:45.062611 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81b9a1-1064-4ed3-a2cb-1bf2a2e7341e None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:45.062852 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81b9a1-1064-4ed3-a2cb-1bf2a2e7341e None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:45.064078 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 289/567] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:45.065328 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81b9a1-1064-4ed3-a2cb-1bf2a2e7341e None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:45.079426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a9ca9-2927-4a7e-9b5b-e2858449bcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:45.080052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a9ca9-2927-4a7e-9b5b-e2858449bcee None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:45.080362 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a9ca9-2927-4a7e-9b5b-e2858449bcee None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:45.080630 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a9ca9-2927-4a7e-9b5b-e2858449bcee None admin] PATH_INFO: `/v3/roles/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:45.091014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a9ca9-2927-4a7e-9b5b-e2858449bcee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:45.096294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-2f4a9ca9-2927-4a7e-9b5b-e2858449bcee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:45.096294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:45.096294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:45.096294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:45.096294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:45.096294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:45.096294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:45.096294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:45.096294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:45.096294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:45.096294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:45.096294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:45.096294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:45.096294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:45.096294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:45.096294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:45.097390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:45.097390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:45.097390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:45.097390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:45.097390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:45.097390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:45.097390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:45.097390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:45.097390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:45.097390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:45.097390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:45.097390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:45.097390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:45.097390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:45.097390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:45.097390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:45.098423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:45.098423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:45.098423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:45.098423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:45.098423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:45.098423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:45.098423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:45.098423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:45.098423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:45.098423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:45.098423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:45.098423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:45.098423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 279/568] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:24:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:45.098423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a9ca9-2927-4a7e-9b5b-e2858449bcee None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:45.123369 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:45.175155 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81b9a1-1064-4ed3-a2cb-1bf2a2e7341e None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:45.193590 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:45.202604 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a9ca9-2927-4a7e-9b5b-e2858449bcee None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:45.208749 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc9e2a-05e2-4b24-b200-20c1350a5a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:45.209369 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc9e2a-05e2-4b24-b200-20c1350a5a5e None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:45.209650 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc9e2a-05e2-4b24-b200-20c1350a5a5e None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:45.209913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc9e2a-05e2-4b24-b200-20c1350a5a5e None admin] PATH_INFO: `/v3/users/nova` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:45.218461 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc9e2a-05e2-4b24-b200-20c1350a5a5e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:45.219897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc9e2a-05e2-4b24-b200-20c1350a5a5e None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:45.232094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-a4fc9e2a-05e2-4b24-b200-20c1350a5a5e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:24:45.232094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:45.232094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:45.232094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:45.232094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:45.232094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:45.232094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:45.232094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:45.232094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:45.232094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:45.232094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:45.232094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:45.232094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:24:45.232094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:24:45.232094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:24:45.232094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:24:45.233216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:45.233216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:45.233216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:24:45.233216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:45.233216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:24:45.233216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:45.233216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:45.233216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:45.233216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:45.233216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:45.233216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:45.233216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:45.233216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:45.233216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:45.233216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:45.233216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:45.233216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:45.233216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:45.234181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:45.234181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:45.234181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:24:45.234181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:24:45.234181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:24:45.234181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:24:45.234181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:24:45.234181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:24:45.234181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:24:45.234181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:45.234181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 290/569] 10.222.0.59 () {56 vars in 1093 bytes} [Mon Aug 29 22:24:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:45.234808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc9e2a-05e2-4b24-b200-20c1350a5a5e None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:45.244048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e8080-8ae7-42b0-b4b5-6bb435378ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:45.244678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e8080-8ae7-42b0-b4b5-6bb435378ceb None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:45.244950 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e8080-8ae7-42b0-b4b5-6bb435378ceb None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:45.245212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e8080-8ae7-42b0-b4b5-6bb435378ceb None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:45.246172 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e8080-8ae7-42b0-b4b5-6bb435378ceb None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:45.246172 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e8080-8ae7-42b0-b4b5-6bb435378ceb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:45.247474 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e8080-8ae7-42b0-b4b5-6bb435378ceb None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:45.259943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 280/570] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:45.261192 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43106e6d-75bc-4dcc-9245-bcf865a82821 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:45.261465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43106e6d-75bc-4dcc-9245-bcf865a82821 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:45.261726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43106e6d-75bc-4dcc-9245-bcf865a82821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:45.279524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-43106e6d-75bc-4dcc-9245-bcf865a82821 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:45.280842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-43106e6d-75bc-4dcc-9245-bcf865a82821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:45.344949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc9e2a-05e2-4b24-b200-20c1350a5a5e None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:45.348256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 281/571] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:45.349123 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43106e6d-75bc-4dcc-9245-bcf865a82821 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:45.359123 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:45.390624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd7cc5-a023-4841-afbd-8db4779165ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:45.392166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd7cc5-a023-4841-afbd-8db4779165ce None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:45.392166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd7cc5-a023-4841-afbd-8db4779165ce None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:45.392166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd7cc5-a023-4841-afbd-8db4779165ce None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:45.394286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd7cc5-a023-4841-afbd-8db4779165ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:45.394286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd7cc5-a023-4841-afbd-8db4779165ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:45.394932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd7cc5-a023-4841-afbd-8db4779165ce None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:45.409897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 291/572] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:24:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:45.410813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd7cc5-a023-4841-afbd-8db4779165ce None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:45.456578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:45.475929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43106e6d-75bc-4dcc-9245-bcf865a82821 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:45.512418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9ea99-c211-4d89-a8a1-723e478a0f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:45.512956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9ea99-c211-4d89-a8a1-723e478a0f07 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:45.513233 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9ea99-c211-4d89-a8a1-723e478a0f07 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:45.513519 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9ea99-c211-4d89-a8a1-723e478a0f07 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:45.514170 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d9ea99-c211-4d89-a8a1-723e478a0f07 None admin] RBAC: Adding query filter params (name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:45.514471 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d9ea99-c211-4d89-a8a1-723e478a0f07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:45.515820 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d9ea99-c211-4d89-a8a1-723e478a0f07 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:45.519256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd7cc5-a023-4841-afbd-8db4779165ce None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:45.525009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 282/573] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:24:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:45.525765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9ea99-c211-4d89-a8a1-723e478a0f07 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:45.558436 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e1310-a56b-4a40-970c-bcab96c13203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:45.559372 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e1310-a56b-4a40-970c-bcab96c13203 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:45.559776 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e1310-a56b-4a40-970c-bcab96c13203 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:45.560248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e1310-a56b-4a40-970c-bcab96c13203 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:45.561137 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e1310-a56b-4a40-970c-bcab96c13203 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:45.561575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e1310-a56b-4a40-970c-bcab96c13203 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:45.563451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e1310-a56b-4a40-970c-bcab96c13203 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:45.587151 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 292/574] 10.222.0.59 () {56 vars in 1133 bytes} [Mon Aug 29 22:24:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:45.587151 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e1310-a56b-4a40-970c-bcab96c13203 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:45.612882 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:45.649939 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9ea99-c211-4d89-a8a1-723e478a0f07 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:45.682347 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107fe64d-9bcb-432e-b630-4e2ac7a42f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:45.682966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fe64d-9bcb-432e-b630-4e2ac7a42f44 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:45.683235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fe64d-9bcb-432e-b630-4e2ac7a42f44 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:45.683502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fe64d-9bcb-432e-b630-4e2ac7a42f44 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:45.688119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e1310-a56b-4a40-970c-bcab96c13203 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:45.690894 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107fe64d-9bcb-432e-b630-4e2ac7a42f44 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:45.692367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107fe64d-9bcb-432e-b630-4e2ac7a42f44 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:45.704038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-107fe64d-9bcb-432e-b630-4e2ac7a42f44 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 22:24:45.704038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:45.704038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:45.704038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:45.704038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:45.704038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:45.704038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:45.704038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:45.704038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:45.704038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:45.704038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:45.704038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:45.704038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:24:45.704038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:24:45.704038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:24:45.704038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:24:45.705210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:45.705210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:45.705210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:24:45.705210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:45.705210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:24:45.705210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:45.705210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:45.705210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:45.705210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:45.705210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:45.705210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:45.705210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:45.705210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:45.705210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:45.705210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:45.705210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:45.705210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:45.705210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:45.706215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:45.706215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:45.706215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:24:45.706215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:24:45.706215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:24:45.706215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:24:45.706215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:24:45.706215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:24:45.706215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 22:24:45.706215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:45.706215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 283/575] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:45.706854 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107fe64d-9bcb-432e-b630-4e2ac7a42f44 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:45.717374 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:45.728023 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c5753-4719-4aa6-88ec-63bceb7b79a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:45.728701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c5753-4719-4aa6-88ec-63bceb7b79a9 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:45.728972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c5753-4719-4aa6-88ec-63bceb7b79a9 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:45.729239 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c5753-4719-4aa6-88ec-63bceb7b79a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:45.729955 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c5753-4719-4aa6-88ec-63bceb7b79a9 None admin] RBAC: Adding query filter params (role.id=f91580fb2e5840bcb4f4054732e2e796, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=f1f0503ae1b24ce6a8adb3cfd7973f14) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:45.730256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c5753-4719-4aa6-88ec-63bceb7b79a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:45.731755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c5753-4719-4aa6-88ec-63bceb7b79a9 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:45.737378 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 293/576] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:24:45 2022] GET /identity/v3/role_assignments?user.id=f1f0503ae1b24ce6a8adb3cfd7973f14&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=f91580fb2e5840bcb4f4054732e2e796 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:45.737378 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c5753-4719-4aa6-88ec-63bceb7b79a9 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:45.828189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107fe64d-9bcb-432e-b630-4e2ac7a42f44 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:45.867710 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24faefd8-f2a6-4568-a978-0b12d206386d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:45.869200 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24faefd8-f2a6-4568-a978-0b12d206386d None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:45.869476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24faefd8-f2a6-4568-a978-0b12d206386d None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:45.869753 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24faefd8-f2a6-4568-a978-0b12d206386d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:45.871347 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:45.882255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24faefd8-f2a6-4568-a978-0b12d206386d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:45.887709 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c5753-4719-4aa6-88ec-63bceb7b79a9 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:45.887852 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24faefd8-f2a6-4568-a978-0b12d206386d None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:45.900965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-24faefd8-f2a6-4568-a978-0b12d206386d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:45.900965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:45.900965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:45.900965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:45.900965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:45.900965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:45.900965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:45.900965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:45.900965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:45.900965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:45.900965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:45.900965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:45.900965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:45.900965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:45.900965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:45.900965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:45.902203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:45.902203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:45.902203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:45.902203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:45.902203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:45.902203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:45.902203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:45.902203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:45.902203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:45.902203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:45.902203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:45.902203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:45.902203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:45.902203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:45.902203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:45.902203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:45.902203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:45.903642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:45.903642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:45.903642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:45.903642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 284/577] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:45.903642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24faefd8-f2a6-4568-a978-0b12d206386d None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:45.938383 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e49b0-d60d-4252-b815-c104c7216a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:45.939194 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e49b0-d60d-4252-b815-c104c7216a41 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:45.939476 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e49b0-d60d-4252-b815-c104c7216a41 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:45.939750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e49b0-d60d-4252-b815-c104c7216a41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:45.950582 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e49b0-d60d-4252-b815-c104c7216a41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:45.952042 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e49b0-d60d-4252-b815-c104c7216a41 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:45.967118 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:45.968636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-0a2e49b0-d60d-4252-b815-c104c7216a41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:45.968636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:45.968636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:45.968636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:45.968636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:45.968636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:45.968636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:45.968636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:45.968636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:45.968636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:45.968636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:45.968636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:45.968636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:45.968636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:45.968636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:45.968636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:45.969569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:45.969569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:45.969569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:45.969569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:45.969569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:45.969569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:45.969569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:45.969569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:45.969569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:45.969569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:45.969569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:45.969569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:45.969569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:45.969569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:45.969569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:45.969569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:45.969569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:45.970620 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:45.970620 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:45.970620 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:45.970620 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 294/578] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:45.971093 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e49b0-d60d-4252-b815-c104c7216a41 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:46.022674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24faefd8-f2a6-4568-a978-0b12d206386d None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:46.059340 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ba41c-4d61-423f-a1f9-6c3f5b03b4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:46.059982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ba41c-4d61-423f-a1f9-6c3f5b03b4bd None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:46.060284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ba41c-4d61-423f-a1f9-6c3f5b03b4bd None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:46.060554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ba41c-4d61-423f-a1f9-6c3f5b03b4bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:46.071185 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ba41c-4d61-423f-a1f9-6c3f5b03b4bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:46.076587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-958ba41c-4d61-423f-a1f9-6c3f5b03b4bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:46.076587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:46.076587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:46.076587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:46.076587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:46.076587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:46.076587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:46.076587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:46.076587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:46.076587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:46.076587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:46.076587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:46.076587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:46.076587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:46.076587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:46.076587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:46.077545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:46.077545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:46.077545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:46.077545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:46.077545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:46.077545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:46.077545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:46.077545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:46.077545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:46.077545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:46.077545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:46.077545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:46.077545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:46.077545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:46.077545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:46.077545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:46.078440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:46.078440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:46.078440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:46.078440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:46.078440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:46.078440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:46.078440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:46.078440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:46.078440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:46.078440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:46.078440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:46.078440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:46.078440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 285/579] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:24:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:46.078440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ba41c-4d61-423f-a1f9-6c3f5b03b4bd None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:46.105637 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e49b0-d60d-4252-b815-c104c7216a41 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:46.147136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed30ce8-42e7-44c0-bcc4-0727dc99484e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:46.147857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed30ce8-42e7-44c0-bcc4-0727dc99484e None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:46.148146 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed30ce8-42e7-44c0-bcc4-0727dc99484e None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:46.148442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed30ce8-42e7-44c0-bcc4-0727dc99484e None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:46.149911 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed30ce8-42e7-44c0-bcc4-0727dc99484e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:46.150342 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed30ce8-42e7-44c0-bcc4-0727dc99484e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:46.152044 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed30ce8-42e7-44c0-bcc4-0727dc99484e None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:46.168324 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:46.172490 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 295/580] 10.222.0.59 () {56 vars in 1137 bytes} [Mon Aug 29 22:24:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:46.173498 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed30ce8-42e7-44c0-bcc4-0727dc99484e None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:46.190011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ba41c-4d61-423f-a1f9-6c3f5b03b4bd None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:46.208043 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:46.231270 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89780a52-9171-4217-881e-53a77e47f1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:46.231889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89780a52-9171-4217-881e-53a77e47f1a1 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:46.232347 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89780a52-9171-4217-881e-53a77e47f1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:46.232693 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89780a52-9171-4217-881e-53a77e47f1a1 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:46.233738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89780a52-9171-4217-881e-53a77e47f1a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:46.234341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89780a52-9171-4217-881e-53a77e47f1a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:46.236246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89780a52-9171-4217-881e-53a77e47f1a1 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:46.255800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 286/581] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:46.255800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89780a52-9171-4217-881e-53a77e47f1a1 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:46.288228 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed30ce8-42e7-44c0-bcc4-0727dc99484e None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:46.327455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aacb91-8173-44c5-aa05-7d3d0d595a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:46.328672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aacb91-8173-44c5-aa05-7d3d0d595a20 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:46.329629 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aacb91-8173-44c5-aa05-7d3d0d595a20 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:46.330150 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aacb91-8173-44c5-aa05-7d3d0d595a20 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:46.331256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aacb91-8173-44c5-aa05-7d3d0d595a20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:46.331916 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aacb91-8173-44c5-aa05-7d3d0d595a20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:46.334375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aacb91-8173-44c5-aa05-7d3d0d595a20 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:46.351574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 296/582] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:46.351574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aacb91-8173-44c5-aa05-7d3d0d595a20 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:46.372560 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89780a52-9171-4217-881e-53a77e47f1a1 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:46.407561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b9823-8a58-4ec9-8552-5a57dbaca3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:46.408319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b9823-8a58-4ec9-8552-5a57dbaca3c1 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:46.408598 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b9823-8a58-4ec9-8552-5a57dbaca3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:46.408865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b9823-8a58-4ec9-8552-5a57dbaca3c1 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:46.409517 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b9823-8a58-4ec9-8552-5a57dbaca3c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:46.409813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b9823-8a58-4ec9-8552-5a57dbaca3c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:46.411161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b9823-8a58-4ec9-8552-5a57dbaca3c1 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:46.418838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:46.420264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 287/583] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:24:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:46.421073 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b9823-8a58-4ec9-8552-5a57dbaca3c1 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:46.466425 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:46.473206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aacb91-8173-44c5-aa05-7d3d0d595a20 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:46.511625 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80798eb-d26e-4c65-931c-60c038444bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:46.512586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80798eb-d26e-4c65-931c-60c038444bcd None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:46.513164 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80798eb-d26e-4c65-931c-60c038444bcd None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:46.513722 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80798eb-d26e-4c65-931c-60c038444bcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:46.526731 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80798eb-d26e-4c65-931c-60c038444bcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:46.528274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80798eb-d26e-4c65-931c-60c038444bcd None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:46.532425 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b9823-8a58-4ec9-8552-5a57dbaca3c1 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:46.543972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-d80798eb-d26e-4c65-931c-60c038444bcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:46.543972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:46.543972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:46.543972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:46.543972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:46.543972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:46.543972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:46.543972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:46.543972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:46.543972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:46.543972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:46.543972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:46.543972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:46.543972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:46.543972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:46.543972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:46.545085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:46.545085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:46.545085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:46.545085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:46.545085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:46.545085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:46.545085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:46.545085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:46.545085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:46.545085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:46.545085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:46.545085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:46.545085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:46.545085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:46.545085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:46.545085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:46.545085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:46.545971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:46.545971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:46.545971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:46.545971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 297/584] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:46.547578 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80798eb-d26e-4c65-931c-60c038444bcd None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:46.576191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfd2da-0dd8-4d26-a38f-7abffb7ea52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:46.576828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfd2da-0dd8-4d26-a38f-7abffb7ea52f None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:46.577104 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfd2da-0dd8-4d26-a38f-7abffb7ea52f None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:46.577363 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfd2da-0dd8-4d26-a38f-7abffb7ea52f None admin] PATH_INFO: `/v3/users/glance` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:46.585554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dfd2da-0dd8-4d26-a38f-7abffb7ea52f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:46.587984 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dfd2da-0dd8-4d26-a38f-7abffb7ea52f None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:46.599909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-c6dfd2da-0dd8-4d26-a38f-7abffb7ea52f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 22:24:46.599909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:46.599909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:46.599909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:46.599909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:46.599909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:46.599909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:46.599909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:46.599909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:46.599909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:46.599909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:46.599909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:46.599909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:24:46.599909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:24:46.599909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:24:46.599909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:24:46.600945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:46.600945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:46.600945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:24:46.600945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:46.600945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:24:46.600945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:46.600945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:46.600945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:46.600945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:46.600945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:46.600945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:46.600945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:46.600945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:46.600945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:46.600945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:46.600945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:46.600945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:46.600945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:46.601933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:46.601933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:46.601933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:24:46.601933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:24:46.601933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:24:46.601933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:24:46.601933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:24:46.601933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:24:46.601933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 22:24:46.601933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:46.601933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 288/585] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:46.602540 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfd2da-0dd8-4d26-a38f-7abffb7ea52f None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:46.677071 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80798eb-d26e-4c65-931c-60c038444bcd None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:46.691116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:46.717545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79507765-1516-4a94-b818-cc01c6d02d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:46.718206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79507765-1516-4a94-b818-cc01c6d02d11 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:46.718456 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79507765-1516-4a94-b818-cc01c6d02d11 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:46.718727 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79507765-1516-4a94-b818-cc01c6d02d11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:46.719774 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:46.726330 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfd2da-0dd8-4d26-a38f-7abffb7ea52f None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:46.726718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79507765-1516-4a94-b818-cc01c6d02d11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:46.728637 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79507765-1516-4a94-b818-cc01c6d02d11 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:46.744861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-79507765-1516-4a94-b818-cc01c6d02d11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:46.744861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:46.744861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:46.744861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:46.744861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:46.744861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:46.744861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:46.744861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:46.744861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:46.744861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:46.744861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:46.744861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:46.744861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:24:46.744861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:24:46.744861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:24:46.746639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:24:46.746639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:46.746639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:46.746639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:46.746639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:46.746639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:46.746639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:46.746639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:46.746639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:46.746639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:46.746639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:46.746639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:46.746639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:46.746639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:46.746639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:46.746639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:46.746639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:46.749004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:24:46.749004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:24:46.749004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:24:46.749004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:24:46.749004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:24:46.749004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:24:46.749004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:46.749004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:46.749004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 298/586] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:46.749004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79507765-1516-4a94-b818-cc01c6d02d11 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:46.776809 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694c152-9360-47f8-8a04-7a2c718f1b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:46.777442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694c152-9360-47f8-8a04-7a2c718f1b85 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:46.777997 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694c152-9360-47f8-8a04-7a2c718f1b85 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:46.777997 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694c152-9360-47f8-8a04-7a2c718f1b85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:46.789223 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694c152-9360-47f8-8a04-7a2c718f1b85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:46.791283 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694c152-9360-47f8-8a04-7a2c718f1b85 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:46.806633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-7694c152-9360-47f8-8a04-7a2c718f1b85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:46.806633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:46.806633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:46.806633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:46.806633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:46.806633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:46.806633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:46.806633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:46.806633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:46.806633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:46.806633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:46.806633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:46.806633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:46.806633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:46.806633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:46.806633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:46.807689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:46.807689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:46.807689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:46.807689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:46.807689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:46.807689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:46.807689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:46.807689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:46.807689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:46.807689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:46.807689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:46.807689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:46.807689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:46.807689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:46.807689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:46.807689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:46.807689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:46.808961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:46.808961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:46.808961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:46.808961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 289/587] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:46.808961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694c152-9360-47f8-8a04-7a2c718f1b85 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:46.879235 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79507765-1516-4a94-b818-cc01c6d02d11 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:46.920775 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f7f89-bda8-45a3-9506-7e84053ea441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:46.921344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f7f89-bda8-45a3-9506-7e84053ea441 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:46.921448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694c152-9360-47f8-8a04-7a2c718f1b85 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:46.921680 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f7f89-bda8-45a3-9506-7e84053ea441 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:46.924565 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f7f89-bda8-45a3-9506-7e84053ea441 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:46.924565 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f7f89-bda8-45a3-9506-7e84053ea441 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:46.924565 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f7f89-bda8-45a3-9506-7e84053ea441 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:46.925133 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f7f89-bda8-45a3-9506-7e84053ea441 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:46.943017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 299/588] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:46.943759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f7f89-bda8-45a3-9506-7e84053ea441 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:46.964535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015ac97-7eee-484d-9422-569e38b26d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:46.965171 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015ac97-7eee-484d-9422-569e38b26d9f None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:46.965446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015ac97-7eee-484d-9422-569e38b26d9f None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:46.965706 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015ac97-7eee-484d-9422-569e38b26d9f None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:46.966561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015ac97-7eee-484d-9422-569e38b26d9f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:46.966911 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015ac97-7eee-484d-9422-569e38b26d9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:46.968360 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015ac97-7eee-484d-9422-569e38b26d9f None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:46.978465 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:46.989320 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 290/589] 10.222.0.59 () {56 vars in 1137 bytes} [Mon Aug 29 22:24:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:46.995464 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015ac97-7eee-484d-9422-569e38b26d9f None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:47.003296 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:47.058798 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f7f89-bda8-45a3-9506-7e84053ea441 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:47.097722 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d2b0e-7a80-4c29-8872-36740b7fc4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:47.098331 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d2b0e-7a80-4c29-8872-36740b7fc4e5 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:47.098609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d2b0e-7a80-4c29-8872-36740b7fc4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:47.098882 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d2b0e-7a80-4c29-8872-36740b7fc4e5 None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:47.099429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015ac97-7eee-484d-9422-569e38b26d9f None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:47.099853 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d2b0e-7a80-4c29-8872-36740b7fc4e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:47.099853 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d2b0e-7a80-4c29-8872-36740b7fc4e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:47.101449 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d2b0e-7a80-4c29-8872-36740b7fc4e5 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:47.115964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 300/590] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:24:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:47.116704 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d2b0e-7a80-4c29-8872-36740b7fc4e5 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:47.147536 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3b229-d1fc-4749-b251-dbb7008c8014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:47.148674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3b229-d1fc-4749-b251-dbb7008c8014 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:47.149675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3b229-d1fc-4749-b251-dbb7008c8014 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:47.150237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3b229-d1fc-4749-b251-dbb7008c8014 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:47.151697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3b229-d1fc-4749-b251-dbb7008c8014 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:47.152277 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3b229-d1fc-4749-b251-dbb7008c8014 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:47.154622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3b229-d1fc-4749-b251-dbb7008c8014 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:47.173931 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 291/591] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:47.175193 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3b229-d1fc-4749-b251-dbb7008c8014 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:47.226378 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:47.237811 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d2b0e-7a80-4c29-8872-36740b7fc4e5 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:47.253816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:47.290700 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3b229-d1fc-4749-b251-dbb7008c8014 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:47.364783 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb6e03-6950-4dbe-9456-901784cf1b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:47.365328 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b82ab0-d5e9-488e-afc3-dda0992a8730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:47.365463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb6e03-6950-4dbe-9456-901784cf1b5d None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:47.365714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb6e03-6950-4dbe-9456-901784cf1b5d None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:47.365812 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b82ab0-d5e9-488e-afc3-dda0992a8730 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:47.365966 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb6e03-6950-4dbe-9456-901784cf1b5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:47.366074 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b82ab0-d5e9-488e-afc3-dda0992a8730 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:47.366336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b82ab0-d5e9-488e-afc3-dda0992a8730 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:47.378384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb6e03-6950-4dbe-9456-901784cf1b5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:47.379852 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb6e03-6950-4dbe-9456-901784cf1b5d None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:47.382895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b82ab0-d5e9-488e-afc3-dda0992a8730 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:47.384975 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b82ab0-d5e9-488e-afc3-dda0992a8730 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:47.393857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-7adb6e03-6950-4dbe-9456-901784cf1b5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:47.393857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:47.393857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:47.393857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:47.393857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:47.393857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:47.393857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:47.393857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:47.393857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:47.393857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:47.393857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:47.393857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:47.393857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:24:47.393857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:24:47.393857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:24:47.394767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:24:47.394767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:47.394767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:47.394767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:47.394767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:47.394767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:47.394767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:47.394767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:47.394767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:47.394767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:47.394767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:47.394767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:47.394767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:47.394767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:47.394767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:47.394767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:47.394767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:47.396098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:24:47.396098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:24:47.396098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:24:47.396098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:24:47.396098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:24:47.396098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:24:47.396098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:47.396098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:47.396098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 301/592] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:47.396870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb6e03-6950-4dbe-9456-901784cf1b5d None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:47.405382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-c6b82ab0-d5e9-488e-afc3-dda0992a8730 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:47.405382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:47.405382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:47.405382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:47.405382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:47.405382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:47.405382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:47.405382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:47.405382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:47.405382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:47.405382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:47.405382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:47.405382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:47.405382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:47.405382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:47.405382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:47.406524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:47.406524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:47.406524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:47.406524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:47.406524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:47.406524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:47.406524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:47.406524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:47.406524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:47.406524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:47.406524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:47.406524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:47.406524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:47.406524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:47.406524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:47.406524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:47.406524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:47.409235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:47.409235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:47.409235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:47.409235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 292/593] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:47.409235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b82ab0-d5e9-488e-afc3-dda0992a8730 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:47.519021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb6e03-6950-4dbe-9456-901784cf1b5d None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:47.523431 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b82ab0-d5e9-488e-afc3-dda0992a8730 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:47.545144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:47.558591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca2373-e228-4cdf-be53-cc00a547dbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:47.559253 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca2373-e228-4cdf-be53-cc00a547dbcb None admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:47.559530 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca2373-e228-4cdf-be53-cc00a547dbcb None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:47.559695 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5d66e-8cf2-483d-abc3-7df7196938c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:47.559815 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca2373-e228-4cdf-be53-cc00a547dbcb None admin] PATH_INFO: `/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/7f21105679f245a19ea4850e6c010dbf/roles/f91580fb2e5840bcb4f4054732e2e796` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:47.560617 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5d66e-8cf2-483d-abc3-7df7196938c6 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:47.561053 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5d66e-8cf2-483d-abc3-7df7196938c6 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:47.561512 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5d66e-8cf2-483d-abc3-7df7196938c6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:47.571432 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5d66e-8cf2-483d-abc3-7df7196938c6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:47.573258 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5d66e-8cf2-483d-abc3-7df7196938c6 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:47.576387 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ca2373-e228-4cdf-be53-cc00a547dbcb None admin] RBAC: Authorizing `identity:create_grant(project_id=7799820fc7c64fd6a99a0a9ccf1338e3, user_id=7f21105679f245a19ea4850e6c010dbf, role_id=f91580fb2e5840bcb4f4054732e2e796)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:47.577899 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ca2373-e228-4cdf-be53-cc00a547dbcb None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:47.586953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-d0c5d66e-8cf2-483d-abc3-7df7196938c6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 22:24:47.586953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:47.586953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:47.586953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:47.586953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:47.586953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:47.586953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:47.586953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:47.586953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:47.586953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:47.586953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:47.586953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:47.586953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:24:47.586953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:24:47.586953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:24:47.586953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:24:47.588442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:47.588442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:47.588442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:24:47.588442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:47.588442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:24:47.588442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:47.588442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:47.588442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:47.588442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:47.588442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:47.588442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:47.588442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:47.588442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:47.588442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:47.588442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:47.588442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:47.588442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:47.588442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:47.589366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:47.589366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:47.589366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:24:47.589366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:24:47.589366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:24:47.589366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:24:47.589366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:24:47.589366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:24:47.589366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 22:24:47.589366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:47.592294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 293/594] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:24:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:47.594317 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5d66e-8cf2-483d-abc3-7df7196938c6 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:47.650714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 302/595] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:24:47 2022] PUT /identity/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/7f21105679f245a19ea4850e6c010dbf/roles/f91580fb2e5840bcb4f4054732e2e796 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:24:47.651657 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca2373-e228-4cdf-be53-cc00a547dbcb None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:47.668543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:47.718114 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5d66e-8cf2-483d-abc3-7df7196938c6 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:47.771650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b417469-0860-4c26-b56a-b456148bc536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:47.772809 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b417469-0860-4c26-b56a-b456148bc536 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:47.773326 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b417469-0860-4c26-b56a-b456148bc536 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:47.774314 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b417469-0860-4c26-b56a-b456148bc536 None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:47.775353 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b417469-0860-4c26-b56a-b456148bc536 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:47.775655 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b417469-0860-4c26-b56a-b456148bc536 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:47.777376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b417469-0860-4c26-b56a-b456148bc536 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:47.794354 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 294/596] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:24:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:47.795402 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b417469-0860-4c26-b56a-b456148bc536 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:47.814636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca2373-e228-4cdf-be53-cc00a547dbcb None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:47.850471 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:47.858836 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4ef03-792f-480a-9205-d2bf9f93413d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:47.859962 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4ef03-792f-480a-9205-d2bf9f93413d None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:47.859962 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4ef03-792f-480a-9205-d2bf9f93413d None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:47.860255 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4ef03-792f-480a-9205-d2bf9f93413d None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:47.860821 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4ef03-792f-480a-9205-d2bf9f93413d None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:47.868090 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4ef03-792f-480a-9205-d2bf9f93413d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:47.868090 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4ef03-792f-480a-9205-d2bf9f93413d None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:47.875855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 303/597] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:47.877151 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028ddce-7229-4ea5-a878-1a8a4d9e2ab9 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:47.877431 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028ddce-7229-4ea5-a878-1a8a4d9e2ab9 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:47.877739 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028ddce-7229-4ea5-a878-1a8a4d9e2ab9 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:47.879094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 304/598] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:47.879597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028ddce-7229-4ea5-a878-1a8a4d9e2ab9 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:47.941525 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b417469-0860-4c26-b56a-b456148bc536 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:47.959657 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:47.981968 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172db97-4eb2-4de5-a3e3-e2184426ee0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:47.982593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172db97-4eb2-4de5-a3e3-e2184426ee0a None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:47.982849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172db97-4eb2-4de5-a3e3-e2184426ee0a None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:47.983111 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172db97-4eb2-4de5-a3e3-e2184426ee0a None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:47.983838 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0172db97-4eb2-4de5-a3e3-e2184426ee0a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:47.984156 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0172db97-4eb2-4de5-a3e3-e2184426ee0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:47.985591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0172db97-4eb2-4de5-a3e3-e2184426ee0a None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:48.005512 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028ddce-7229-4ea5-a878-1a8a4d9e2ab9 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:48.005937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 295/599] 10.222.0.59 () {56 vars in 1139 bytes} [Mon Aug 29 22:24:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:48.011397 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172db97-4eb2-4de5-a3e3-e2184426ee0a None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:48.052088 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304de0f-b1e2-464d-b3dc-92b1fa4b4caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:48.053059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304de0f-b1e2-464d-b3dc-92b1fa4b4caf None admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:48.053585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304de0f-b1e2-464d-b3dc-92b1fa4b4caf None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:48.054087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304de0f-b1e2-464d-b3dc-92b1fa4b4caf None admin] PATH_INFO: `/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/92af56c889e640e7815c1641c05a9af3/roles/f91580fb2e5840bcb4f4054732e2e796` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:48.072387 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304de0f-b1e2-464d-b3dc-92b1fa4b4caf None admin] RBAC: Authorizing `identity:create_grant(project_id=7799820fc7c64fd6a99a0a9ccf1338e3, user_id=92af56c889e640e7815c1641c05a9af3, role_id=f91580fb2e5840bcb4f4054732e2e796)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:48.074029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304de0f-b1e2-464d-b3dc-92b1fa4b4caf None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:48.140663 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:48.145625 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172db97-4eb2-4de5-a3e3-e2184426ee0a None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:48.150202 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 305/600] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:24:47 2022] PUT /identity/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/92af56c889e640e7815c1641c05a9af3/roles/f91580fb2e5840bcb4f4054732e2e796 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:24:48.151649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcb2d0-7479-4bc3-9c6f-7fed9c2d3841 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:48.152128 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcb2d0-7479-4bc3-9c6f-7fed9c2d3841 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:48.152582 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcb2d0-7479-4bc3-9c6f-7fed9c2d3841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:48.175380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-00dcb2d0-7479-4bc3-9c6f-7fed9c2d3841 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:48.176972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-00dcb2d0-7479-4bc3-9c6f-7fed9c2d3841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:48.198733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f5bfd-47ee-4612-825f-156ff6aaf74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:48.199452 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f5bfd-47ee-4612-825f-156ff6aaf74d None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:48.199789 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f5bfd-47ee-4612-825f-156ff6aaf74d None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:48.200108 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f5bfd-47ee-4612-825f-156ff6aaf74d None admin] PATH_INFO: `/v3/projects/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:48.216455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f5bfd-47ee-4612-825f-156ff6aaf74d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:48.219441 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f5bfd-47ee-4612-825f-156ff6aaf74d None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:48.237238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-6f8f5bfd-47ee-4612-825f-156ff6aaf74d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:48.237238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:48.237238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:48.237238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:48.237238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:48.237238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:48.237238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:48.237238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:48.237238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:48.237238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:48.237238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:48.237238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:48.237238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:24:48.237238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:24:48.237238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:24:48.240924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:24:48.240924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:48.240924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:48.240924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:48.240924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:48.240924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:48.240924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:48.240924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:48.240924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:48.240924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:48.240924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:48.240924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:48.240924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:48.240924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:48.240924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:48.240924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:48.240924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:48.241798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:24:48.241798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:24:48.241798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:24:48.241798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:24:48.241798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:24:48.241798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:24:48.241798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:48.241798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:48.241798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 296/601] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:48.244031 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f5bfd-47ee-4612-825f-156ff6aaf74d None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:48.284186 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 306/602] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:48.284186 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcb2d0-7479-4bc3-9c6f-7fed9c2d3841 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:48.359357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:48.402863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f5bfd-47ee-4612-825f-156ff6aaf74d None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:48.441613 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcb2d0-7479-4bc3-9c6f-7fed9c2d3841 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:48.461005 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bdd9a-32d5-46a6-ba7c-aaa140711a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:48.462571 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bdd9a-32d5-46a6-ba7c-aaa140711a08 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:48.462946 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bdd9a-32d5-46a6-ba7c-aaa140711a08 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:48.463314 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bdd9a-32d5-46a6-ba7c-aaa140711a08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:48.483922 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bdd9a-32d5-46a6-ba7c-aaa140711a08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:48.485569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bdd9a-32d5-46a6-ba7c-aaa140711a08 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:48.513698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-4b3bdd9a-32d5-46a6-ba7c-aaa140711a08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:48.513698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:48.513698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:48.513698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:48.513698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:48.513698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:48.513698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:48.513698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:48.513698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:48.513698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:48.513698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:48.513698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:48.513698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:48.513698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:48.513698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:48.513698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:48.514872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:48.514872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:48.514872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:48.514872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:48.514872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:48.514872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:48.514872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:48.514872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:48.514872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:48.514872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:48.514872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:48.514872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:48.514872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:48.514872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:48.514872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:48.514872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:48.514872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:48.515940 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:48.515940 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:48.515940 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:48.518034 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 297/603] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:48.525243 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046f77c-20b2-4599-81cf-626ee2f07a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:48.525992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046f77c-20b2-4599-81cf-626ee2f07a1f None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:48.526389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046f77c-20b2-4599-81cf-626ee2f07a1f None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:48.526792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046f77c-20b2-4599-81cf-626ee2f07a1f None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:48.527601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046f77c-20b2-4599-81cf-626ee2f07a1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:48.528091 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046f77c-20b2-4599-81cf-626ee2f07a1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:48.529651 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046f77c-20b2-4599-81cf-626ee2f07a1f None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:48.532361 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bdd9a-32d5-46a6-ba7c-aaa140711a08 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:48.551071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:48.557423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 307/604] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:24:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:48.561379 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046f77c-20b2-4599-81cf-626ee2f07a1f None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:48.684065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bdd9a-32d5-46a6-ba7c-aaa140711a08 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:48.734754 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046f77c-20b2-4599-81cf-626ee2f07a1f None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:48.737199 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e2cc0-0fee-42a5-bd56-68adc7cfaaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:48.738918 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e2cc0-0fee-42a5-bd56-68adc7cfaaf3 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:48.739298 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e2cc0-0fee-42a5-bd56-68adc7cfaaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:48.739580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e2cc0-0fee-42a5-bd56-68adc7cfaaf3 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:48.740458 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e2cc0-0fee-42a5-bd56-68adc7cfaaf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:48.740592 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e2cc0-0fee-42a5-bd56-68adc7cfaaf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:48.741918 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e2cc0-0fee-42a5-bd56-68adc7cfaaf3 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:48.744186 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:48.757584 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 298/605] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:48.758923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e2cc0-0fee-42a5-bd56-68adc7cfaaf3 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:48.791629 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100059a-6220-403a-92b9-7c2f5690adf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:48.792410 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100059a-6220-403a-92b9-7c2f5690adf8 None admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:48.792754 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100059a-6220-403a-92b9-7c2f5690adf8 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:48.793105 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100059a-6220-403a-92b9-7c2f5690adf8 None admin] PATH_INFO: `/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/4db08aa2e4ef4478a3909a876754814c/roles/f91580fb2e5840bcb4f4054732e2e796` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:48.813091 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100059a-6220-403a-92b9-7c2f5690adf8 None admin] RBAC: Authorizing `identity:create_grant(project_id=7799820fc7c64fd6a99a0a9ccf1338e3, user_id=4db08aa2e4ef4478a3909a876754814c, role_id=f91580fb2e5840bcb4f4054732e2e796)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:48.814643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100059a-6220-403a-92b9-7c2f5690adf8 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:48.844995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:48.885204 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e2cc0-0fee-42a5-bd56-68adc7cfaaf3 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:48.894216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 308/606] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:24:48 2022] PUT /identity/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/4db08aa2e4ef4478a3909a876754814c/roles/f91580fb2e5840bcb4f4054732e2e796 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:24:48.895676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100059a-6220-403a-92b9-7c2f5690adf8 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:48.953668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca4742-5133-4c15-8762-39f9aaf5033a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:48.954620 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca4742-5133-4c15-8762-39f9aaf5033a None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:48.954902 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca4742-5133-4c15-8762-39f9aaf5033a None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:48.955168 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca4742-5133-4c15-8762-39f9aaf5033a None admin] PATH_INFO: `/v3/roles/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:48.987880 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca4742-5133-4c15-8762-39f9aaf5033a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:48.997913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-48ca4742-5133-4c15-8762-39f9aaf5033a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:48.997913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:48.997913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:48.997913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:48.997913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:48.997913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:48.997913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:48.997913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:48.997913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:48.997913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:48.997913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:48.997913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:48.997913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:48.997913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:48.997913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:48.997913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:48.998904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:48.998904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:48.998904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:48.998904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:48.998904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:48.998904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:48.998904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:48.998904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:48.998904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:48.998904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:48.998904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:48.998904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:48.998904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:48.998904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:48.998904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:48.998904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:48.999739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:48.999739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:48.999739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:48.999739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:48.999739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:48.999739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:48.999739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:48.999739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:48.999739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:48.999739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:48.999739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:48.999739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:49.000399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 299/607] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:24:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:49.010986 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca4742-5133-4c15-8762-39f9aaf5033a None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:49.116420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100059a-6220-403a-92b9-7c2f5690adf8 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:49.138347 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca4742-5133-4c15-8762-39f9aaf5033a None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:49.170371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:49.176507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c7e8c-04c1-4782-bff8-05de211b1658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:49.177121 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c7e8c-04c1-4782-bff8-05de211b1658 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:49.177396 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c7e8c-04c1-4782-bff8-05de211b1658 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:49.177660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c7e8c-04c1-4782-bff8-05de211b1658 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:49.178524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c7e8c-04c1-4782-bff8-05de211b1658 None admin] RBAC: Adding query filter params (name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:49.178828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c7e8c-04c1-4782-bff8-05de211b1658 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:49.180117 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe0b736-7b9f-46d9-992b-0fd3b0f219e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:49.181112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0b736-7b9f-46d9-992b-0fd3b0f219e8 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:49.181240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c7e8c-04c1-4782-bff8-05de211b1658 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:49.181709 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0b736-7b9f-46d9-992b-0fd3b0f219e8 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:49.182180 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0b736-7b9f-46d9-992b-0fd3b0f219e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:49.189659 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe0b736-7b9f-46d9-992b-0fd3b0f219e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:49.190949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 300/608] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:24:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:49.203762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe0b736-7b9f-46d9-992b-0fd3b0f219e8 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:49.203904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c7e8c-04c1-4782-bff8-05de211b1658 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:49.205013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-bfe0b736-7b9f-46d9-992b-0fd3b0f219e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:49.205013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:49.205013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:49.205013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:49.205013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:49.205013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:49.205013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:49.205013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:49.205013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:49.205013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:49.205013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:49.205013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:49.205013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:24:49.205013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:24:49.205013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:24:49.205860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:24:49.205860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:49.205860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:49.205860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:49.205860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:49.205860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:49.205860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:49.205860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:49.205860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:49.205860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:49.205860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:49.205860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:49.205860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:49.205860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:49.205860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:49.205860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:49.205860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:49.206779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:24:49.206779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:24:49.206779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:24:49.206779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:24:49.206779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:24:49.206779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:24:49.206779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:49.206779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:49.209241 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 309/609] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:49.215622 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe0b736-7b9f-46d9-992b-0fd3b0f219e8 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:49.238660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:49.334573 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c7e8c-04c1-4782-bff8-05de211b1658 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:49.379962 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f1634-98db-4671-9d97-b374b9143949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:49.381054 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f1634-98db-4671-9d97-b374b9143949 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:49.382369 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f1634-98db-4671-9d97-b374b9143949 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:49.384730 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f1634-98db-4671-9d97-b374b9143949 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:49.389180 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe0b736-7b9f-46d9-992b-0fd3b0f219e8 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:49.410453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f1634-98db-4671-9d97-b374b9143949 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:49.412318 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f1634-98db-4671-9d97-b374b9143949 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:49.446815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-4a3f1634-98db-4671-9d97-b374b9143949 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:49.446815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:49.446815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:49.446815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:49.446815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:49.446815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:49.446815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:49.446815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:49.446815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:49.446815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:49.446815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:49.446815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:49.446815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:49.446815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:49.446815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:49.446815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:49.448563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:49.448563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:49.448563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:49.448563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:49.448563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:49.448563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:49.448563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:49.448563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:49.448563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:49.448563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:49.448563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:49.448563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:49.448563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:49.448563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:49.448563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:49.448563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:49.448563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:49.450198 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:49.450198 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:49.450198 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:49.454467 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 301/610] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:24:49.460971 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f1634-98db-4671-9d97-b374b9143949 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:49.485801 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df968fb-ad70-409e-8af0-0ab0136363d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:49.487726 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df968fb-ad70-409e-8af0-0ab0136363d3 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:49.489540 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df968fb-ad70-409e-8af0-0ab0136363d3 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:49.490027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df968fb-ad70-409e-8af0-0ab0136363d3 None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:49.491173 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df968fb-ad70-409e-8af0-0ab0136363d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:49.493935 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df968fb-ad70-409e-8af0-0ab0136363d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:49.497112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df968fb-ad70-409e-8af0-0ab0136363d3 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:49.531063 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 310/611] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:24:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:49.536750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df968fb-ad70-409e-8af0-0ab0136363d3 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:49.558721 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:49.630084 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f1634-98db-4671-9d97-b374b9143949 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:49.675088 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531c65a-16d4-46d3-95c4-2cc51322a1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:49.676346 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531c65a-16d4-46d3-95c4-2cc51322a1b8 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:49.678113 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531c65a-16d4-46d3-95c4-2cc51322a1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:49.678891 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531c65a-16d4-46d3-95c4-2cc51322a1b8 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:49.680394 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531c65a-16d4-46d3-95c4-2cc51322a1b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:49.681835 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531c65a-16d4-46d3-95c4-2cc51322a1b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:49.684899 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531c65a-16d4-46d3-95c4-2cc51322a1b8 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:49.710738 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:49.715375 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 302/612] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:49.722318 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531c65a-16d4-46d3-95c4-2cc51322a1b8 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:49.788911 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df968fb-ad70-409e-8af0-0ab0136363d3 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:49.836766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbe08d-9061-469c-92f4-217413106698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:49.838388 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbe08d-9061-469c-92f4-217413106698 None admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:49.839259 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbe08d-9061-469c-92f4-217413106698 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:49.840008 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbe08d-9061-469c-92f4-217413106698 None admin] PATH_INFO: `/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/cb802d01e71b486cb40c6388430d674d/roles/f91580fb2e5840bcb4f4054732e2e796` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:49.855426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531c65a-16d4-46d3-95c4-2cc51322a1b8 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:49.858215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:49.870925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cbe08d-9061-469c-92f4-217413106698 None admin] RBAC: Authorizing `identity:create_grant(project_id=7799820fc7c64fd6a99a0a9ccf1338e3, user_id=cb802d01e71b486cb40c6388430d674d, role_id=f91580fb2e5840bcb4f4054732e2e796)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:49.873983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cbe08d-9061-469c-92f4-217413106698 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:49.896793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81a00a-bd82-44a4-8aee-2ca2ac64e7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:49.897528 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81a00a-bd82-44a4-8aee-2ca2ac64e7c3 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:49.897888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81a00a-bd82-44a4-8aee-2ca2ac64e7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:49.898223 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81a00a-bd82-44a4-8aee-2ca2ac64e7c3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:49.907785 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81a00a-bd82-44a4-8aee-2ca2ac64e7c3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:49.909447 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81a00a-bd82-44a4-8aee-2ca2ac64e7c3 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:49.923466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-0c81a00a-bd82-44a4-8aee-2ca2ac64e7c3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:24:49.923466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:49.923466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:49.923466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:49.923466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:49.923466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:49.923466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:49.923466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:49.923466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:49.923466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:49.923466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:49.923466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:49.923466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:24:49.923466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:24:49.923466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:24:49.923466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:24:49.924507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:49.924507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:49.924507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:24:49.924507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:49.924507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:24:49.924507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:49.924507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:49.924507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:49.924507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:49.924507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:49.924507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:49.924507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:49.924507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:49.924507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:49.924507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:49.924507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:49.924507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:49.924507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:49.925456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:49.925456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:49.925456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:24:49.925456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:24:49.925456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:24:49.925456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:24:49.925456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:24:49.925456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:24:49.925456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:24:49.925456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:49.929162 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 303/613] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:24:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:24:49.931377 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3d8f8-7012-4e4c-9ced-db7411adbf1b None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:49.931710 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3d8f8-7012-4e4c-9ced-db7411adbf1b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:49.931986 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3d8f8-7012-4e4c-9ced-db7411adbf1b None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:49.933515 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 304/614] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:49.933977 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3d8f8-7012-4e4c-9ced-db7411adbf1b None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:49.947956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 311/615] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:24:49 2022] PUT /identity/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/cb802d01e71b486cb40c6388430d674d/roles/f91580fb2e5840bcb4f4054732e2e796 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:24:49.949679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341edf7-be69-490e-969a-a861f781a8a7 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:49.950101 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341edf7-be69-490e-969a-a861f781a8a7 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:49.950518 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341edf7-be69-490e-969a-a861f781a8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:49.974475 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-4341edf7-be69-490e-969a-a861f781a8a7 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:49.976191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-4341edf7-be69-490e-969a-a861f781a8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:50.067953 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 312/616] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:50.135590 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3d8f8-7012-4e4c-9ced-db7411adbf1b None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:50.176386 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6204b2c-1f13-4426-96fa-78d06a8709cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:50.178118 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6204b2c-1f13-4426-96fa-78d06a8709cd None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:50.178454 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6204b2c-1f13-4426-96fa-78d06a8709cd None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:50.178730 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6204b2c-1f13-4426-96fa-78d06a8709cd None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:50.179486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6204b2c-1f13-4426-96fa-78d06a8709cd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:50.179789 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6204b2c-1f13-4426-96fa-78d06a8709cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:50.182531 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6204b2c-1f13-4426-96fa-78d06a8709cd None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:50.204658 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 305/617] 10.222.0.59 () {56 vars in 1135 bytes} [Mon Aug 29 22:24:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:50.210229 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341edf7-be69-490e-969a-a861f781a8a7 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:50.278528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:50.334923 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341edf7-be69-490e-969a-a861f781a8a7 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:50.373055 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24319b-4ac8-4319-bd7e-6a84a9055f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:50.373805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24319b-4ac8-4319-bd7e-6a84a9055f59 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:50.374084 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24319b-4ac8-4319-bd7e-6a84a9055f59 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:50.374355 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24319b-4ac8-4319-bd7e-6a84a9055f59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:50.378833 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6204b2c-1f13-4426-96fa-78d06a8709cd None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:50.387423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24319b-4ac8-4319-bd7e-6a84a9055f59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:50.389500 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24319b-4ac8-4319-bd7e-6a84a9055f59 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:50.407151 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:50.412193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-de24319b-4ac8-4319-bd7e-6a84a9055f59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:50.412193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:50.412193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:50.412193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:50.412193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:50.412193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:50.412193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:50.412193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:50.412193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:50.412193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:50.412193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:50.412193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:50.412193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:50.412193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:50.412193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:50.412193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:50.417368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:50.417368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:50.417368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:50.417368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:50.417368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:50.417368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:50.417368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:50.417368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:50.417368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:50.417368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:50.417368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:50.417368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:50.417368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:50.417368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:50.417368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:50.417368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:50.417368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:50.418258 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:50.418258 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:50.418258 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:50.418258 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 313/618] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:24:50.420649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24319b-4ac8-4319-bd7e-6a84a9055f59 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:50.556666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6204b2c-1f13-4426-96fa-78d06a8709cd None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:50.581823 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24319b-4ac8-4319-bd7e-6a84a9055f59 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:50.585221 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:50.603968 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11e15e-acba-4875-a5d3-86a2f4496273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:50.605103 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11e15e-acba-4875-a5d3-86a2f4496273 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:50.605584 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11e15e-acba-4875-a5d3-86a2f4496273 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:50.605982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11e15e-acba-4875-a5d3-86a2f4496273 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:50.619502 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c60e2-1906-40c8-9a15-41a6a075c78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:50.620315 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c60e2-1906-40c8-9a15-41a6a075c78d None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:50.620690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c60e2-1906-40c8-9a15-41a6a075c78d None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:50.621083 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c60e2-1906-40c8-9a15-41a6a075c78d None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:50.621895 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c60e2-1906-40c8-9a15-41a6a075c78d None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:50.622290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c60e2-1906-40c8-9a15-41a6a075c78d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:50.622676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11e15e-acba-4875-a5d3-86a2f4496273 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:50.623887 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c60e2-1906-40c8-9a15-41a6a075c78d None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:50.628864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-8d11e15e-acba-4875-a5d3-86a2f4496273 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:50.628864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:50.628864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:50.628864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:50.628864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:50.628864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:50.628864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:50.628864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:50.628864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:50.628864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:50.628864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:50.628864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:50.628864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:50.628864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:50.628864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:50.628864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:50.630900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:50.630900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:50.630900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:50.630900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:50.630900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:50.630900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:50.630900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:50.630900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:50.630900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:50.630900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:50.630900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:50.630900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:50.630900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:50.630900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:50.630900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:50.630900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:50.632679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:50.632679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:50.632679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:50.632679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:50.632679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:50.632679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:50.632679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:50.632679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:50.632679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:50.632679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:50.632679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:50.632679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:50.632679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 306/619] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:24:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:50.632679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719a2d9-5ccc-42ee-afae-d23f22839806 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:50.633697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719a2d9-5ccc-42ee-afae-d23f22839806 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:50.633697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719a2d9-5ccc-42ee-afae-d23f22839806 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:50.638761 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 307/620] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:50.644186 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5b9c7-d304-41bc-8c86-bbdfcf4910a3 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:50.644186 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5b9c7-d304-41bc-8c86-bbdfcf4910a3 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:50.644186 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5b9c7-d304-41bc-8c86-bbdfcf4910a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:50.644538 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 314/621] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:50.644692 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c60e2-1906-40c8-9a15-41a6a075c78d None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:50.661245 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-13a5b9c7-d304-41bc-8c86-bbdfcf4910a3 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:50.662637 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-13a5b9c7-d304-41bc-8c86-bbdfcf4910a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:50.732892 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 308/622] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:50.733949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5b9c7-d304-41bc-8c86-bbdfcf4910a3 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:50.752973 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:50.810166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c60e2-1906-40c8-9a15-41a6a075c78d None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:50.863892 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e924546-a204-4da5-ae5f-ed31d275c68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:50.864492 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e924546-a204-4da5-ae5f-ed31d275c68c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:50.864768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e924546-a204-4da5-ae5f-ed31d275c68c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:50.865037 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e924546-a204-4da5-ae5f-ed31d275c68c None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:50.865696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e924546-a204-4da5-ae5f-ed31d275c68c None admin] RBAC: Adding query filter params (name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:50.866416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e924546-a204-4da5-ae5f-ed31d275c68c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:50.868166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e924546-a204-4da5-ae5f-ed31d275c68c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:50.878503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 315/623] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:24:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:50.884876 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e924546-a204-4da5-ae5f-ed31d275c68c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:50.928640 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5b9c7-d304-41bc-8c86-bbdfcf4910a3 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:50.930054 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:51.004879 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158589d8-b6a5-4b1a-ae9f-a17f8baaa6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:51.005549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158589d8-b6a5-4b1a-ae9f-a17f8baaa6de None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:51.005832 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158589d8-b6a5-4b1a-ae9f-a17f8baaa6de None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:51.006081 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158589d8-b6a5-4b1a-ae9f-a17f8baaa6de None admin] PATH_INFO: `/v3/projects/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:51.012449 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e924546-a204-4da5-ae5f-ed31d275c68c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:51.014127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158589d8-b6a5-4b1a-ae9f-a17f8baaa6de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:51.015948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158589d8-b6a5-4b1a-ae9f-a17f8baaa6de None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:51.035674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-158589d8-b6a5-4b1a-ae9f-a17f8baaa6de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:51.035674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:51.035674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:51.035674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:51.035674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:51.035674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:51.035674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:51.035674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:51.035674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:51.035674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:51.035674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:51.035674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:51.035674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:24:51.035674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:24:51.035674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:24:51.036853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:24:51.036853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:51.036853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:51.036853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:51.036853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:51.036853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:51.036853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:51.036853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:51.036853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:51.036853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:51.036853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:51.036853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:51.036853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:51.036853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:51.036853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:51.036853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:51.036853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:51.037857 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:24:51.037857 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:24:51.037857 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:24:51.037857 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:24:51.037857 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:24:51.037857 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:24:51.037857 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:51.037857 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:51.044118 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 309/624] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:51.045591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158589d8-b6a5-4b1a-ae9f-a17f8baaa6de None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:51.055510 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b889aa-9a28-4a48-b755-2b551741a1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:51.056509 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b889aa-9a28-4a48-b755-2b551741a1db None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:51.056789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b889aa-9a28-4a48-b755-2b551741a1db None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:51.057060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b889aa-9a28-4a48-b755-2b551741a1db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:51.068274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b889aa-9a28-4a48-b755-2b551741a1db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:51.069728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b889aa-9a28-4a48-b755-2b551741a1db None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:51.084947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-c4b889aa-9a28-4a48-b755-2b551741a1db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:51.084947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:51.084947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:51.084947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:51.084947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:51.084947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:51.084947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:51.084947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:51.084947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:51.084947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:51.084947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:51.084947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:51.084947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:51.084947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:51.084947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:51.084947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:51.085914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:51.085914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:51.085914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:51.085914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:51.085914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:51.085914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:51.085914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:51.085914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:51.085914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:51.085914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:51.085914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:51.085914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:51.085914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:51.085914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:51.085914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:51.085914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:51.085914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:51.086865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:51.086865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:51.086865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:51.086865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 316/625] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:51.091771 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b889aa-9a28-4a48-b755-2b551741a1db None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:51.135014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:51.172271 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158589d8-b6a5-4b1a-ae9f-a17f8baaa6de None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:51.173031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:51.198318 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b889aa-9a28-4a48-b755-2b551741a1db None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:51.212166 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c3920-1e53-47e6-806e-7a7c886bd4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:51.213850 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c3920-1e53-47e6-806e-7a7c886bd4f8 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:51.214292 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c3920-1e53-47e6-806e-7a7c886bd4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:51.214806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c3920-1e53-47e6-806e-7a7c886bd4f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:51.226199 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c3920-1e53-47e6-806e-7a7c886bd4f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:51.232600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-1c5c3920-1e53-47e6-806e-7a7c886bd4f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:51.232600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:51.232600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:51.232600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:51.232600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:51.232600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:51.232600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:51.232600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:51.232600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:51.232600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:51.232600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:51.232600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:51.232600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:51.232600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:51.232600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:51.232600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:51.233800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:51.233800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:51.233800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:51.233800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:51.233800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:51.233800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:51.233800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:51.233800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:51.233800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:51.233800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:51.233800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:51.233800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:51.233800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:51.233800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:51.233800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:51.233800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:51.234831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:51.234831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:51.234831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:51.234831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:51.234831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:51.234831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:51.234831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:51.234831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:51.234831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:51.234831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:51.234831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:51.234831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:51.237112 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 310/626] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:24:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:51.237794 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc0b81-914b-4ba0-9370-174d81f6f859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:51.238261 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c3920-1e53-47e6-806e-7a7c886bd4f8 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:51.238470 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc0b81-914b-4ba0-9370-174d81f6f859 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:51.238750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc0b81-914b-4ba0-9370-174d81f6f859 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:51.239070 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc0b81-914b-4ba0-9370-174d81f6f859 None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:51.239768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc0b81-914b-4ba0-9370-174d81f6f859 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:51.240095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc0b81-914b-4ba0-9370-174d81f6f859 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:51.241604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc0b81-914b-4ba0-9370-174d81f6f859 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:51.257037 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 317/627] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:24:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:51.257958 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc0b81-914b-4ba0-9370-174d81f6f859 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:51.355737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c3920-1e53-47e6-806e-7a7c886bd4f8 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:51.366750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc0b81-914b-4ba0-9370-174d81f6f859 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:51.394021 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:51.404491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b58432-8ce4-4ebc-b3de-8c7b10d62efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:51.406110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b58432-8ce4-4ebc-b3de-8c7b10d62efc None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:51.406573 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b58432-8ce4-4ebc-b3de-8c7b10d62efc None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:51.407085 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b58432-8ce4-4ebc-b3de-8c7b10d62efc None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:51.408790 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b58432-8ce4-4ebc-b3de-8c7b10d62efc None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:51.409804 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b58432-8ce4-4ebc-b3de-8c7b10d62efc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:51.411561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b58432-8ce4-4ebc-b3de-8c7b10d62efc None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:51.414234 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce474f2e-3bf1-47ef-a7d9-dd9eb1bad178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:51.415436 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce474f2e-3bf1-47ef-a7d9-dd9eb1bad178 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:51.415784 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce474f2e-3bf1-47ef-a7d9-dd9eb1bad178 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:51.416586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce474f2e-3bf1-47ef-a7d9-dd9eb1bad178 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:51.416932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce474f2e-3bf1-47ef-a7d9-dd9eb1bad178 None admin] RBAC: Adding query filter params (name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:51.417303 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce474f2e-3bf1-47ef-a7d9-dd9eb1bad178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:51.418766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce474f2e-3bf1-47ef-a7d9-dd9eb1bad178 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:51.425767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:51.429599 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 318/628] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:24:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:51.430268 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce474f2e-3bf1-47ef-a7d9-dd9eb1bad178 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:51.432648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 311/629] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:51.441148 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b58432-8ce4-4ebc-b3de-8c7b10d62efc None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:51.553809 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce474f2e-3bf1-47ef-a7d9-dd9eb1bad178 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:51.579442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b58432-8ce4-4ebc-b3de-8c7b10d62efc None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:51.592115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf17e2c-305d-4839-900b-8792e8826574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:51.593161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf17e2c-305d-4839-900b-8792e8826574 None admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:51.593394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf17e2c-305d-4839-900b-8792e8826574 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:51.593674 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf17e2c-305d-4839-900b-8792e8826574 None admin] PATH_INFO: `/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/f1f0503ae1b24ce6a8adb3cfd7973f14/roles/f91580fb2e5840bcb4f4054732e2e796` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:51.612531 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf17e2c-305d-4839-900b-8792e8826574 None admin] RBAC: Authorizing `identity:create_grant(project_id=7799820fc7c64fd6a99a0a9ccf1338e3, user_id=f1f0503ae1b24ce6a8adb3cfd7973f14, role_id=f91580fb2e5840bcb4f4054732e2e796)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:51.614116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf17e2c-305d-4839-900b-8792e8826574 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:51.618954 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63170806-3ca6-4389-9127-de516667fbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:51.619582 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63170806-3ca6-4389-9127-de516667fbed None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:51.619855 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63170806-3ca6-4389-9127-de516667fbed None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:51.620116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63170806-3ca6-4389-9127-de516667fbed None admin] PATH_INFO: `/v3/users/nova` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:51.627357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63170806-3ca6-4389-9127-de516667fbed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:51.628814 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63170806-3ca6-4389-9127-de516667fbed None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:51.640991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-63170806-3ca6-4389-9127-de516667fbed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:24:51.640991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:51.640991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:51.640991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:51.640991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:51.640991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:51.640991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:51.640991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:51.640991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:51.640991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:51.640991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:51.640991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:51.640991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:24:51.640991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:24:51.640991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:24:51.640991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:24:51.642129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:51.642129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:51.642129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:24:51.642129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:51.642129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:24:51.642129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:51.642129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:51.642129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:51.642129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:51.642129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:51.642129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:51.642129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:51.642129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:51.642129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:51.642129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:51.642129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:51.642129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:51.642129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:51.643518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:51.643518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:51.643518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:24:51.643518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:24:51.643518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:24:51.643518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:24:51.643518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:24:51.643518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:24:51.643518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:24:51.643518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:51.643518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 312/630] 10.222.0.59 () {56 vars in 1093 bytes} [Mon Aug 29 22:24:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:51.643518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63170806-3ca6-4389-9127-de516667fbed None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:51.698482 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:51.700334 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 319/631] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:24:51 2022] PUT /identity/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/f1f0503ae1b24ce6a8adb3cfd7973f14/roles/f91580fb2e5840bcb4f4054732e2e796 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:24:51.701954 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c15fae-8f3e-45d4-916f-0d1ac14ea486 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:51.702241 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c15fae-8f3e-45d4-916f-0d1ac14ea486 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:51.702513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c15fae-8f3e-45d4-916f-0d1ac14ea486 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:51.704036 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 320/632] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:51.709385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c15fae-8f3e-45d4-916f-0d1ac14ea486 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:51.785545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63170806-3ca6-4389-9127-de516667fbed None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:51.808174 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:51.832673 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc99d85-1930-4e78-b39c-d721788ca8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:51.833904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc99d85-1930-4e78-b39c-d721788ca8f8 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:51.834291 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc99d85-1930-4e78-b39c-d721788ca8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:51.835844 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc99d85-1930-4e78-b39c-d721788ca8f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:51.849442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc99d85-1930-4e78-b39c-d721788ca8f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:51.851558 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc99d85-1930-4e78-b39c-d721788ca8f8 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:51.870687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-9dc99d85-1930-4e78-b39c-d721788ca8f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:51.870687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:51.870687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:51.870687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:51.870687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:51.870687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:51.870687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:51.870687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:51.870687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:51.870687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:51.870687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:51.870687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:51.870687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:51.870687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:51.870687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:51.870687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:51.871929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:51.871929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:51.871929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:51.871929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:51.871929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:51.871929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:51.871929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:51.871929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:51.871929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:51.871929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:51.871929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:51.871929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:51.871929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:51.871929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:51.871929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:51.871929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:51.871929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:51.873164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:51.873164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:51.873164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:51.873164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 313/633] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:51.874306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31665f-261f-4fbe-a810-929c2c643000 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:51.874306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31665f-261f-4fbe-a810-929c2c643000 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:51.874480 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31665f-261f-4fbe-a810-929c2c643000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:51.879353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c15fae-8f3e-45d4-916f-0d1ac14ea486 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:51.899710 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-af31665f-261f-4fbe-a810-929c2c643000 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:51.902563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-af31665f-261f-4fbe-a810-929c2c643000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:51.936859 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b737a7a-3e05-4c6f-8a6f-7f228030d257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:51.937636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b737a7a-3e05-4c6f-8a6f-7f228030d257 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:51.938019 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b737a7a-3e05-4c6f-8a6f-7f228030d257 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:51.938396 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b737a7a-3e05-4c6f-8a6f-7f228030d257 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:51.939425 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b737a7a-3e05-4c6f-8a6f-7f228030d257 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:51.939847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b737a7a-3e05-4c6f-8a6f-7f228030d257 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:51.941458 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b737a7a-3e05-4c6f-8a6f-7f228030d257 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:51.962357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 321/634] 10.222.0.59 () {56 vars in 1133 bytes} [Mon Aug 29 22:24:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:51.968162 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b737a7a-3e05-4c6f-8a6f-7f228030d257 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:51.993562 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 314/635] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:51.994664 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31665f-261f-4fbe-a810-929c2c643000 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:52.032466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:52.080653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b737a7a-3e05-4c6f-8a6f-7f228030d257 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:52.093455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:52.122970 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784271ae-4ecb-4e10-986a-07723b454aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:52.123920 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784271ae-4ecb-4e10-986a-07723b454aad None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:52.124366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784271ae-4ecb-4e10-986a-07723b454aad None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:52.125238 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784271ae-4ecb-4e10-986a-07723b454aad None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:52.126459 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784271ae-4ecb-4e10-986a-07723b454aad None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:52.126916 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784271ae-4ecb-4e10-986a-07723b454aad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:52.128608 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784271ae-4ecb-4e10-986a-07723b454aad None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:52.142135 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 322/636] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:52.150611 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784271ae-4ecb-4e10-986a-07723b454aad None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:52.183908 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31665f-261f-4fbe-a810-929c2c643000 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:52.225008 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4218b-4a46-4767-aaf5-013435ca3093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:52.225658 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4218b-4a46-4767-aaf5-013435ca3093 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:52.226029 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4218b-4a46-4767-aaf5-013435ca3093 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:52.226361 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4218b-4a46-4767-aaf5-013435ca3093 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:52.267637 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784271ae-4ecb-4e10-986a-07723b454aad None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:52.308104 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308205b-c8c9-4675-800f-9ee8db14a863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:52.309091 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308205b-c8c9-4675-800f-9ee8db14a863 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:52.309530 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308205b-c8c9-4675-800f-9ee8db14a863 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:52.309946 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308205b-c8c9-4675-800f-9ee8db14a863 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:52.319135 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308205b-c8c9-4675-800f-9ee8db14a863 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:52.321077 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308205b-c8c9-4675-800f-9ee8db14a863 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:52.333879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-d308205b-c8c9-4675-800f-9ee8db14a863 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 22:24:52.333879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:52.333879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:52.333879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:52.333879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:52.333879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:52.333879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:52.333879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:52.333879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:52.333879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:52.333879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:52.333879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:52.333879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:24:52.333879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:24:52.333879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:24:52.333879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:24:52.335100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:52.335100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:52.335100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:24:52.335100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:52.335100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:24:52.335100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:52.335100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:52.335100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:52.335100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:52.335100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:52.335100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:52.335100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:52.335100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:52.335100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:52.335100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:52.335100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:52.335100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:52.335100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:52.337001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:52.337001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:52.337001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:24:52.337001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:24:52.337001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:24:52.337001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:24:52.337001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:24:52.337001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:24:52.337001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 22:24:52.337001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:52.339105 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 323/637] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:52.345173 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308205b-c8c9-4675-800f-9ee8db14a863 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:52.354263 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4218b-4a46-4767-aaf5-013435ca3093 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:52.356001 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4218b-4a46-4767-aaf5-013435ca3093 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:52.394362 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:52.411646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:52.472740 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308205b-c8c9-4675-800f-9ee8db14a863 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:52.484880 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 315/638] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:52.484880 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4218b-4a46-4767-aaf5-013435ca3093 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:52.519452 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dfeacb-7e28-40ca-9e0b-c58881ad83b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:52.520248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfeacb-7e28-40ca-9e0b-c58881ad83b8 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:52.520626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfeacb-7e28-40ca-9e0b-c58881ad83b8 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:52.520996 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfeacb-7e28-40ca-9e0b-c58881ad83b8 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:52.521828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dfeacb-7e28-40ca-9e0b-c58881ad83b8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:52.522236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dfeacb-7e28-40ca-9e0b-c58881ad83b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:52.524382 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dfeacb-7e28-40ca-9e0b-c58881ad83b8 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:52.545933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 324/639] 10.222.0.59 () {56 vars in 1137 bytes} [Mon Aug 29 22:24:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:52.547541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8bc72-6f86-4429-8761-ebb022613e2c None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:52.547920 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8bc72-6f86-4429-8761-ebb022613e2c None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:52.548399 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8bc72-6f86-4429-8761-ebb022613e2c None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:52.553610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 325/640] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:52.553610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8bc72-6f86-4429-8761-ebb022613e2c None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:52.615156 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4218b-4a46-4767-aaf5-013435ca3093 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:52.651586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:52.652979 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb2292-8f7d-44d6-b55a-6448086851bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:52.653732 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb2292-8f7d-44d6-b55a-6448086851bc None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:52.654108 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb2292-8f7d-44d6-b55a-6448086851bc None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:52.654483 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb2292-8f7d-44d6-b55a-6448086851bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:52.656890 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:52.667613 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb2292-8f7d-44d6-b55a-6448086851bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:52.668053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8bc72-6f86-4429-8761-ebb022613e2c None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:52.672648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-2bcb2292-8f7d-44d6-b55a-6448086851bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:52.672648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:52.672648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:52.672648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:52.672648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:52.672648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:52.672648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:52.672648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:52.672648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:52.672648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:52.672648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:52.672648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:52.672648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:52.672648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:52.672648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:52.672648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:52.673942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:52.673942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:52.673942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:52.673942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:52.673942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:52.673942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:52.673942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:52.673942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:52.673942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:52.673942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:52.673942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:52.673942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:52.673942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:52.673942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:52.673942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:52.673942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:52.675010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:52.675010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:52.675010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:52.675010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:52.675010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:52.675010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:52.675010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:52.675010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:52.675010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:52.675010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:52.675010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:52.675010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:52.675725 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 316/641] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:24:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:52.676495 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb2292-8f7d-44d6-b55a-6448086851bc None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:52.709369 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b52275-fc5a-48d4-bb7e-4008dea22487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:52.709997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b52275-fc5a-48d4-bb7e-4008dea22487 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:52.710273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b52275-fc5a-48d4-bb7e-4008dea22487 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:52.710545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b52275-fc5a-48d4-bb7e-4008dea22487 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:52.721632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b52275-fc5a-48d4-bb7e-4008dea22487 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:52.723073 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b52275-fc5a-48d4-bb7e-4008dea22487 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:52.738453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-a7b52275-fc5a-48d4-bb7e-4008dea22487 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:52.738453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:52.738453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:52.738453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:52.738453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:52.738453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:52.738453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:52.738453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:52.738453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:52.738453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:52.738453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:52.738453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:52.738453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:52.738453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:52.738453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:52.738453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:52.739464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:52.739464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:52.739464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:52.739464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:52.739464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:52.739464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:52.739464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:52.739464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:52.739464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:52.739464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:52.739464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:52.739464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:52.739464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:52.739464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:52.739464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:52.739464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:52.739464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:52.740390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:52.740390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:52.740390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:52.740390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 326/642] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:52.740975 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19410c-875d-4d8e-91d4-83a5dc60e90f None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:52.740975 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19410c-875d-4d8e-91d4-83a5dc60e90f None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:52.741179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19410c-875d-4d8e-91d4-83a5dc60e90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:52.760613 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-5f19410c-875d-4d8e-91d4-83a5dc60e90f None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:52.762343 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-5f19410c-875d-4d8e-91d4-83a5dc60e90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:52.797862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb2292-8f7d-44d6-b55a-6448086851bc None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:52.864327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e5b01-d92a-400e-8635-1066babf9991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:52.864551 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 327/643] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:52.864551 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f19410c-875d-4d8e-91d4-83a5dc60e90f None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:52.864755 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e5b01-d92a-400e-8635-1066babf9991 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:52.864755 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e5b01-d92a-400e-8635-1066babf9991 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:52.864755 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e5b01-d92a-400e-8635-1066babf9991 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:52.956201 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:52.958991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:52.970593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e5b01-d92a-400e-8635-1066babf9991 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:52.973091 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e5b01-d92a-400e-8635-1066babf9991 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:53.005079 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f19410c-875d-4d8e-91d4-83a5dc60e90f None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:53.055619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf05ff-0de3-4a48-9b58-89a45ae8ebc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:53.056823 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf05ff-0de3-4a48-9b58-89a45ae8ebc8 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:53.057205 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf05ff-0de3-4a48-9b58-89a45ae8ebc8 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:53.057574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf05ff-0de3-4a48-9b58-89a45ae8ebc8 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:53.058595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf05ff-0de3-4a48-9b58-89a45ae8ebc8 None admin] RBAC: Adding query filter params (name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:53.059401 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf05ff-0de3-4a48-9b58-89a45ae8ebc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:53.061228 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf05ff-0de3-4a48-9b58-89a45ae8ebc8 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:53.073310 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 328/644] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:24:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:53.074538 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf05ff-0de3-4a48-9b58-89a45ae8ebc8 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:53.114008 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 317/645] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:53.114008 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e5b01-d92a-400e-8635-1066babf9991 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:53.192178 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf05ff-0de3-4a48-9b58-89a45ae8ebc8 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:53.223702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:53.227384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:53.233428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e5b01-d92a-400e-8635-1066babf9991 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:53.237443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2559fbd-1c00-4f49-8b60-7ac3b8e53f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:53.238053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2559fbd-1c00-4f49-8b60-7ac3b8e53f63 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:53.238325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2559fbd-1c00-4f49-8b60-7ac3b8e53f63 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:53.238589 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2559fbd-1c00-4f49-8b60-7ac3b8e53f63 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:53.239231 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2559fbd-1c00-4f49-8b60-7ac3b8e53f63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:53.239528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2559fbd-1c00-4f49-8b60-7ac3b8e53f63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:53.240925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2559fbd-1c00-4f49-8b60-7ac3b8e53f63 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:53.257513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 329/646] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:53.259163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2559fbd-1c00-4f49-8b60-7ac3b8e53f63 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:53.282497 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6072d9-027e-4a95-9696-12cb01829031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:53.283160 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6072d9-027e-4a95-9696-12cb01829031 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:53.283441 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6072d9-027e-4a95-9696-12cb01829031 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:53.283715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6072d9-027e-4a95-9696-12cb01829031 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:53.294711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6072d9-027e-4a95-9696-12cb01829031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:53.296175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6072d9-027e-4a95-9696-12cb01829031 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:53.311180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-cd6072d9-027e-4a95-9696-12cb01829031 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:53.311180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:53.311180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:53.311180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:53.311180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:53.311180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:53.311180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:53.311180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:53.311180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:53.311180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:53.311180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:53.311180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:53.311180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:53.311180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:53.311180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:53.311180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:53.312329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:53.312329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:53.312329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:53.312329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:53.312329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:53.312329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:53.312329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:53.312329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:53.312329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:53.312329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:53.312329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:53.312329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:53.312329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:53.312329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:53.312329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:53.312329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:53.312329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:53.313293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:53.313293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:53.313293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:53.317136 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 318/647] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:53.317742 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6072d9-027e-4a95-9696-12cb01829031 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:53.370809 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2559fbd-1c00-4f49-8b60-7ac3b8e53f63 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:53.409616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311172a-3158-431d-8972-fb977f1074ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:53.410246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311172a-3158-431d-8972-fb977f1074ff None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:53.410530 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311172a-3158-431d-8972-fb977f1074ff None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:53.411238 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311172a-3158-431d-8972-fb977f1074ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:53.422622 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311172a-3158-431d-8972-fb977f1074ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:53.424548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311172a-3158-431d-8972-fb977f1074ff None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:53.437038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6072d9-027e-4a95-9696-12cb01829031 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:53.440779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-7311172a-3158-431d-8972-fb977f1074ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:53.440779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:53.440779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:53.440779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:53.440779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:53.440779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:53.440779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:53.440779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:53.440779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:53.440779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:53.440779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:53.440779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:53.440779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:53.440779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:53.440779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:53.440779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:53.442412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:53.442412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:53.442412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:53.442412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:53.442412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:53.442412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:53.442412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:53.442412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:53.442412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:53.442412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:53.442412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:53.442412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:53.442412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:53.442412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:53.442412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:53.442412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:53.442412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:53.443828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:53.443828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:53.443828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:53.443828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 330/648] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:53.443828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311172a-3158-431d-8972-fb977f1074ff None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:53.488714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9711dd2-2e6d-43da-a0f3-8f6c40216c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:53.489382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9711dd2-2e6d-43da-a0f3-8f6c40216c0c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:53.489663 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9711dd2-2e6d-43da-a0f3-8f6c40216c0c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:53.489935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9711dd2-2e6d-43da-a0f3-8f6c40216c0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:53.504092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9711dd2-2e6d-43da-a0f3-8f6c40216c0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:53.507257 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:53.514498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-e9711dd2-2e6d-43da-a0f3-8f6c40216c0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:53.514498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:53.514498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:53.514498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:53.514498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:53.514498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:53.514498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:53.514498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:53.514498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:53.514498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:53.514498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:53.514498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:53.514498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:53.514498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:53.514498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:53.514498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:53.516260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:53.516260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:53.516260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:53.516260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:53.516260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:53.516260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:53.516260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:53.516260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:53.516260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:53.516260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:53.516260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:53.516260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:53.516260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:53.516260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:53.516260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:53.516260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:53.517863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:53.517863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:53.517863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:53.517863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:53.517863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:53.517863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:53.517863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:53.517863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:53.517863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:53.517863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:53.517863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:53.517863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:53.520363 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 319/649] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:24:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:53.522783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9711dd2-2e6d-43da-a0f3-8f6c40216c0c None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:53.541212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:53.588205 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311172a-3158-431d-8972-fb977f1074ff None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:53.630122 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69f190-a886-4f09-9690-790ff92ff287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:53.631075 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69f190-a886-4f09-9690-790ff92ff287 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:53.631196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69f190-a886-4f09-9690-790ff92ff287 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:53.631479 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69f190-a886-4f09-9690-790ff92ff287 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:53.640035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69f190-a886-4f09-9690-790ff92ff287 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:53.641603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69f190-a886-4f09-9690-790ff92ff287 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:53.654370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-3d69f190-a886-4f09-9690-790ff92ff287 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:53.654370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:53.654370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:53.654370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:53.654370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:53.654370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:53.654370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:53.654370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:53.654370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:53.654370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:53.654370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:53.654370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:53.654370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:24:53.654370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:24:53.654370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:24:53.655718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:24:53.655718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:53.655718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:53.655718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:53.655718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:53.655718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:53.655718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:53.655718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:53.655718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:53.655718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:53.655718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:53.655718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:53.655718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:53.655718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:53.655718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:53.655718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:53.655718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:53.656656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:24:53.656656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:24:53.656656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:24:53.656656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:24:53.656656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:24:53.656656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:24:53.656656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:53.656656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:53.656656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 331/650] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:53.656656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69f190-a886-4f09-9690-790ff92ff287 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:53.668604 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9711dd2-2e6d-43da-a0f3-8f6c40216c0c None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:53.703920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c96ac-79de-4483-985b-f4e8aadc9c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:53.704656 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c96ac-79de-4483-985b-f4e8aadc9c41 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:53.704994 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c96ac-79de-4483-985b-f4e8aadc9c41 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:53.705270 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c96ac-79de-4483-985b-f4e8aadc9c41 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:53.705914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c96ac-79de-4483-985b-f4e8aadc9c41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:53.706231 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c96ac-79de-4483-985b-f4e8aadc9c41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:53.707616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c96ac-79de-4483-985b-f4e8aadc9c41 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:53.722462 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 320/651] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:53.723607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c96ac-79de-4483-985b-f4e8aadc9c41 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:53.774385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69f190-a886-4f09-9690-790ff92ff287 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:53.776119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:53.793644 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:53.814776 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3c664-0c44-4f58-8661-04a9970a37ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:53.815386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3c664-0c44-4f58-8661-04a9970a37ec None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:53.815726 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3c664-0c44-4f58-8661-04a9970a37ec None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:53.816439 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3c664-0c44-4f58-8661-04a9970a37ec None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:53.817097 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3c664-0c44-4f58-8661-04a9970a37ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:53.817678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3c664-0c44-4f58-8661-04a9970a37ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:53.819250 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3c664-0c44-4f58-8661-04a9970a37ec None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:53.828085 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c96ac-79de-4483-985b-f4e8aadc9c41 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:53.840020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 332/652] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:53.840020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3c664-0c44-4f58-8661-04a9970a37ec None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:53.864820 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858752c-9136-4cc1-b786-1f66f332c781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:53.865606 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858752c-9136-4cc1-b786-1f66f332c781 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:53.865886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858752c-9136-4cc1-b786-1f66f332c781 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:53.866152 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858752c-9136-4cc1-b786-1f66f332c781 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:53.866843 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858752c-9136-4cc1-b786-1f66f332c781 None admin] RBAC: Adding query filter params (name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:53.867116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858752c-9136-4cc1-b786-1f66f332c781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:53.868761 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858752c-9136-4cc1-b786-1f66f332c781 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:53.877260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 321/653] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:24:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:53.877906 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858752c-9136-4cc1-b786-1f66f332c781 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:53.971965 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3c664-0c44-4f58-8661-04a9970a37ec None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:53.991487 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858752c-9136-4cc1-b786-1f66f332c781 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:54.006611 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5fbaa-8852-4a2c-979b-ac981373a095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:54.007204 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5fbaa-8852-4a2c-979b-ac981373a095 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:54.007476 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5fbaa-8852-4a2c-979b-ac981373a095 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:54.008525 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5fbaa-8852-4a2c-979b-ac981373a095 None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:54.009166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5fbaa-8852-4a2c-979b-ac981373a095 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:54.009464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5fbaa-8852-4a2c-979b-ac981373a095 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:54.010923 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5fbaa-8852-4a2c-979b-ac981373a095 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:54.018446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:54.029167 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 333/654] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:24:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:54.029167 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5fbaa-8852-4a2c-979b-ac981373a095 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:54.029432 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0261892-4938-4644-b16f-bfb837dc54f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:54.029432 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0261892-4938-4644-b16f-bfb837dc54f8 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:54.029432 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0261892-4938-4644-b16f-bfb837dc54f8 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:54.029432 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0261892-4938-4644-b16f-bfb837dc54f8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:54.036492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0261892-4938-4644-b16f-bfb837dc54f8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:54.037912 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0261892-4938-4644-b16f-bfb837dc54f8 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:54.049061 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:54.050350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-b0261892-4938-4644-b16f-bfb837dc54f8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 22:24:54.050350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:54.050350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:54.050350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:54.050350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:54.050350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:54.050350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:54.050350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:54.050350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:54.050350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:54.050350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:54.050350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:54.050350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:24:54.050350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:24:54.050350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:24:54.050350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:24:54.051477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:54.051477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:54.051477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:24:54.051477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:54.051477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:24:54.051477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:54.051477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:54.051477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:54.051477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:54.051477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:54.051477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:54.051477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:54.051477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:54.051477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:54.051477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:54.051477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:54.051477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:54.051477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:54.052491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:54.052491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:54.052491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:24:54.052491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:24:54.052491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:24:54.052491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:24:54.052491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:24:54.052491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:24:54.052491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 22:24:54.052491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:54.052491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 322/655] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:54.053093 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0261892-4938-4644-b16f-bfb837dc54f8 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:54.129164 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5fbaa-8852-4a2c-979b-ac981373a095 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:54.155796 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0261892-4938-4644-b16f-bfb837dc54f8 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:54.164566 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e948c-6925-443e-9f39-00355a824a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:54.165459 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e948c-6925-443e-9f39-00355a824a86 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:54.165739 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e948c-6925-443e-9f39-00355a824a86 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:54.166002 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e948c-6925-443e-9f39-00355a824a86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:54.172699 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e948c-6925-443e-9f39-00355a824a86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:54.174158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e948c-6925-443e-9f39-00355a824a86 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:54.186384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-6b7e948c-6925-443e-9f39-00355a824a86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:54.186384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:54.186384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:54.186384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:54.186384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:54.186384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:54.186384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:54.186384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:54.186384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:54.186384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:54.186384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:54.186384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:54.186384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:24:54.186384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:24:54.186384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:24:54.187345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:24:54.187345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:54.187345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:54.187345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:54.187345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:54.187345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:54.187345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:54.187345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:54.187345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:54.187345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:54.187345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:54.187345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:54.187345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:54.187345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:54.187345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:54.187345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:54.187345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:54.188847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:24:54.188847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:24:54.188847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:24:54.188847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:24:54.188847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:24:54.188847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:24:54.188847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:54.188847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:54.188847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 334/656] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:54.188847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d0a71-24b4-46b4-bdcb-580e7ef3682f None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:54.189530 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d0a71-24b4-46b4-bdcb-580e7ef3682f None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:54.189530 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d0a71-24b4-46b4-bdcb-580e7ef3682f None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:54.190352 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 335/657] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:54.191022 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d0a71-24b4-46b4-bdcb-580e7ef3682f None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:54.196338 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a89c2-777e-4b16-953f-85d2c7e0390a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:54.197247 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a89c2-777e-4b16-953f-85d2c7e0390a None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:54.197462 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a89c2-777e-4b16-953f-85d2c7e0390a None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:54.199122 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a89c2-777e-4b16-953f-85d2c7e0390a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:54.209764 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a89c2-777e-4b16-953f-85d2c7e0390a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:54.212093 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a89c2-777e-4b16-953f-85d2c7e0390a None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:54.226259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-397a89c2-777e-4b16-953f-85d2c7e0390a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:54.226259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:54.226259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:54.226259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:54.226259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:54.226259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:54.226259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:54.226259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:54.226259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:54.226259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:54.226259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:54.226259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:54.226259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:54.226259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:54.226259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:54.226259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:54.227442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:54.227442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:54.227442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:54.227442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:54.227442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:54.227442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:54.227442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:54.227442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:54.227442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:54.227442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:54.227442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:54.227442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:54.227442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:54.227442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:54.227442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:54.227442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:54.227442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:54.228354 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:54.228354 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:54.228354 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:54.228354 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 323/658] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:54.228871 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a89c2-777e-4b16-953f-85d2c7e0390a None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:54.272124 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:54.286270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:54.305198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d0a71-24b4-46b4-bdcb-580e7ef3682f None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:54.329347 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a89c2-777e-4b16-953f-85d2c7e0390a None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:54.340943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8849a0-cb55-4d38-b7fd-2ec07b8aa3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:54.341543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8849a0-cb55-4d38-b7fd-2ec07b8aa3ca None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:54.341816 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8849a0-cb55-4d38-b7fd-2ec07b8aa3ca None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:54.342081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8849a0-cb55-4d38-b7fd-2ec07b8aa3ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:54.342800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8849a0-cb55-4d38-b7fd-2ec07b8aa3ca None admin] RBAC: Adding query filter params (role.id=f91580fb2e5840bcb4f4054732e2e796, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=7f21105679f245a19ea4850e6c010dbf) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:54.343098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8849a0-cb55-4d38-b7fd-2ec07b8aa3ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:54.344537 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8849a0-cb55-4d38-b7fd-2ec07b8aa3ca None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:54.350107 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 336/659] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:24:54 2022] GET /identity/v3/role_assignments?user.id=7f21105679f245a19ea4850e6c010dbf&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=f91580fb2e5840bcb4f4054732e2e796 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:54.350868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8849a0-cb55-4d38-b7fd-2ec07b8aa3ca None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:54.374063 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5f494-ab61-4f0a-9aaa-f1382bcbda7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:54.374798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5f494-ab61-4f0a-9aaa-f1382bcbda7f None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:54.375074 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5f494-ab61-4f0a-9aaa-f1382bcbda7f None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:54.375342 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5f494-ab61-4f0a-9aaa-f1382bcbda7f None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:54.376096 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e5f494-ab61-4f0a-9aaa-f1382bcbda7f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:54.376431 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e5f494-ab61-4f0a-9aaa-f1382bcbda7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:54.377901 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e5f494-ab61-4f0a-9aaa-f1382bcbda7f None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:54.398775 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 324/660] 10.222.0.59 () {56 vars in 1137 bytes} [Mon Aug 29 22:24:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:54.399840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044f269-8380-442a-9465-596685b2dea7 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:54.400237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044f269-8380-442a-9465-596685b2dea7 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:54.400579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044f269-8380-442a-9465-596685b2dea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:54.419830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-4044f269-8380-442a-9465-596685b2dea7 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:54.421246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-4044f269-8380-442a-9465-596685b2dea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:54.475391 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8849a0-cb55-4d38-b7fd-2ec07b8aa3ca None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:54.503275 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 325/661] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:54.504252 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044f269-8380-442a-9465-596685b2dea7 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:54.518551 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3dba0-11b4-4c1e-8510-69d19a4ed2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:54.519157 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3dba0-11b4-4c1e-8510-69d19a4ed2ff None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:54.519439 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3dba0-11b4-4c1e-8510-69d19a4ed2ff None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:54.519710 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3dba0-11b4-4c1e-8510-69d19a4ed2ff None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:54.520434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3dba0-11b4-4c1e-8510-69d19a4ed2ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:54.520739 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3dba0-11b4-4c1e-8510-69d19a4ed2ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:54.522166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3dba0-11b4-4c1e-8510-69d19a4ed2ff None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:54.534851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 337/662] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:24:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:54.535679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3dba0-11b4-4c1e-8510-69d19a4ed2ff None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:54.545807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:54.546733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:54.639726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044f269-8380-442a-9465-596685b2dea7 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:54.673773 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3dba0-11b4-4c1e-8510-69d19a4ed2ff None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:54.676369 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd34329-44a6-4fd4-a14a-9e812c811f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:54.677083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd34329-44a6-4fd4-a14a-9e812c811f21 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:54.677413 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd34329-44a6-4fd4-a14a-9e812c811f21 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:54.677738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd34329-44a6-4fd4-a14a-9e812c811f21 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:54.678397 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd34329-44a6-4fd4-a14a-9e812c811f21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:54.678699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd34329-44a6-4fd4-a14a-9e812c811f21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:54.680358 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd34329-44a6-4fd4-a14a-9e812c811f21 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:54.693005 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 326/663] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:54.693674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd34329-44a6-4fd4-a14a-9e812c811f21 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:54.718932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4a0af-1aa0-4b59-8979-739f54f00dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:54.719592 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4a0af-1aa0-4b59-8979-739f54f00dcc None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:54.719811 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4a0af-1aa0-4b59-8979-739f54f00dcc None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:54.720105 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4a0af-1aa0-4b59-8979-739f54f00dcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:54.800046 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4a0af-1aa0-4b59-8979-739f54f00dcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:54.801857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4a0af-1aa0-4b59-8979-739f54f00dcc None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:54.811255 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:54.812571 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:54.826415 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd34329-44a6-4fd4-a14a-9e812c811f21 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:54.861278 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620e508-baab-4590-9107-b10c32dda054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:54.862034 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620e508-baab-4590-9107-b10c32dda054 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:54.862312 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620e508-baab-4590-9107-b10c32dda054 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:54.862582 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620e508-baab-4590-9107-b10c32dda054 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:54.863298 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620e508-baab-4590-9107-b10c32dda054 None admin] RBAC: Adding query filter params (role.id=f91580fb2e5840bcb4f4054732e2e796, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=92af56c889e640e7815c1641c05a9af3) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:54.863608 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620e508-baab-4590-9107-b10c32dda054 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:54.865173 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620e508-baab-4590-9107-b10c32dda054 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:54.876465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 327/664] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:24:54 2022] GET /identity/v3/role_assignments?user.id=92af56c889e640e7815c1641c05a9af3&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=f91580fb2e5840bcb4f4054732e2e796 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:54.877238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620e508-baab-4590-9107-b10c32dda054 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:54.920374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 338/665] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:54.920374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4a0af-1aa0-4b59-8979-739f54f00dcc None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:55.016652 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620e508-baab-4590-9107-b10c32dda054 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:55.041838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4a0af-1aa0-4b59-8979-739f54f00dcc None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:55.050538 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:55.057486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6eb55-c106-42e8-8e7b-1e1cab893607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:55.062985 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6eb55-c106-42e8-8e7b-1e1cab893607 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:55.062985 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6eb55-c106-42e8-8e7b-1e1cab893607 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:55.062985 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6eb55-c106-42e8-8e7b-1e1cab893607 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:55.068001 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6eb55-c106-42e8-8e7b-1e1cab893607 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:55.069627 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6eb55-c106-42e8-8e7b-1e1cab893607 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:55.081093 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b0d4c-7e3a-46f8-a2dc-2d7e445822c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:55.081821 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b0d4c-7e3a-46f8-a2dc-2d7e445822c7 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:55.082172 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b0d4c-7e3a-46f8-a2dc-2d7e445822c7 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:55.082450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b0d4c-7e3a-46f8-a2dc-2d7e445822c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:55.083400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-2fe6eb55-c106-42e8-8e7b-1e1cab893607 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 22:24:55.083400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:55.083400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:55.083400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:55.083400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:55.083400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:55.083400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:55.083400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:55.083400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:55.083400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:55.083400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:55.083400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:55.083400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:24:55.083400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:24:55.083400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:24:55.083400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:24:55.084334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:55.084334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:55.084334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:24:55.084334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:55.084334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:24:55.084334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:55.084334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:55.084334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:55.084334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:55.084334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:55.084334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:55.084334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:55.084334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:55.084334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:55.084334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:55.084334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:55.084334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:55.084334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:55.085284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:55.085284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:55.085284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:24:55.085284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:24:55.085284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:24:55.085284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:24:55.085284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:24:55.085284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:24:55.085284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 22:24:55.085284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:55.085284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 328/666] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:24:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:55.085284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6eb55-c106-42e8-8e7b-1e1cab893607 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:55.100351 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b0d4c-7e3a-46f8-a2dc-2d7e445822c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:55.105464 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:55.108074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-b64b0d4c-7e3a-46f8-a2dc-2d7e445822c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:55.108074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:55.108074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:55.108074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:55.108074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:55.108074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:55.108074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:55.108074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:55.108074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:55.108074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:55.108074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:55.108074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:55.108074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:55.108074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:55.108074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:55.108074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:55.109069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:55.109069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:55.109069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:55.109069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:55.109069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:55.109069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:55.109069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:55.109069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:55.109069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:55.109069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:55.109069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:55.109069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:55.109069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:55.109069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:55.109069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:55.109069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:55.110380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:55.110380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:55.110380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:55.110380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:55.110380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:55.110380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:55.110380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:55.110380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:55.110380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:55.110380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:55.110380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:24:55.110380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:55.110380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 339/667] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:24:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:55.110380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b0d4c-7e3a-46f8-a2dc-2d7e445822c7 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:55.231672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6eb55-c106-42e8-8e7b-1e1cab893607 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:55.264634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b0d4c-7e3a-46f8-a2dc-2d7e445822c7 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:55.279918 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4eaeec-0004-41ca-8a09-8ee251dbda83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:55.280653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4eaeec-0004-41ca-8a09-8ee251dbda83 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:55.280883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4eaeec-0004-41ca-8a09-8ee251dbda83 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:55.281154 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4eaeec-0004-41ca-8a09-8ee251dbda83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:55.297364 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4eaeec-0004-41ca-8a09-8ee251dbda83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:55.299598 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4eaeec-0004-41ca-8a09-8ee251dbda83 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:55.308442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6d708-e37a-425b-8fcc-21ce33c55b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:55.309390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6d708-e37a-425b-8fcc-21ce33c55b27 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:55.309867 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6d708-e37a-425b-8fcc-21ce33c55b27 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:55.310374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6d708-e37a-425b-8fcc-21ce33c55b27 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:55.311352 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6d708-e37a-425b-8fcc-21ce33c55b27 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:55.311868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6d708-e37a-425b-8fcc-21ce33c55b27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:55.313666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6d708-e37a-425b-8fcc-21ce33c55b27 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:55.318357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-9c4eaeec-0004-41ca-8a09-8ee251dbda83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:55.318357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:55.318357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:55.318357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:55.318357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:55.318357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:55.318357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:55.318357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:55.318357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:55.318357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:55.318357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:55.318357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:55.318357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:55.318357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:55.318357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:55.318357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:55.319344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:55.319344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:55.319344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:55.319344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:55.319344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:55.319344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:55.319344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:55.319344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:55.319344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:55.319344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:55.319344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:55.319344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:55.319344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:55.319344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:55.319344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:55.319344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:55.319344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:55.320396 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:55.320396 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:55.320396 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:55.320396 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 329/668] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:55.324507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4eaeec-0004-41ca-8a09-8ee251dbda83 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:55.339323 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 340/669] 10.222.0.59 () {56 vars in 1139 bytes} [Mon Aug 29 22:24:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:55.340583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6d708-e37a-425b-8fcc-21ce33c55b27 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:55.371766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:55.415952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:55.468192 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4eaeec-0004-41ca-8a09-8ee251dbda83 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:55.472488 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6d708-e37a-425b-8fcc-21ce33c55b27 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:55.507941 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96d964-78ee-499a-b32a-badb064a111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:55.508711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96d964-78ee-499a-b32a-badb064a111e None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:55.508993 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96d964-78ee-499a-b32a-badb064a111e None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:55.509265 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96d964-78ee-499a-b32a-badb064a111e None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:55.509942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a96d964-78ee-499a-b32a-badb064a111e None admin] RBAC: Adding query filter params (name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:55.510241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a96d964-78ee-499a-b32a-badb064a111e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:55.511710 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a96d964-78ee-499a-b32a-badb064a111e None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:55.513379 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0682c-1936-4382-b086-7df8a6fafd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:55.514251 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0682c-1936-4382-b086-7df8a6fafd31 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:55.514738 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0682c-1936-4382-b086-7df8a6fafd31 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:55.515204 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0682c-1936-4382-b086-7df8a6fafd31 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:55.516050 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0682c-1936-4382-b086-7df8a6fafd31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:55.516572 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0682c-1936-4382-b086-7df8a6fafd31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:55.518140 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0682c-1936-4382-b086-7df8a6fafd31 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:55.521587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 330/670] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:24:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:55.522442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96d964-78ee-499a-b32a-badb064a111e None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:55.532548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 341/671] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:55.533723 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0682c-1936-4382-b086-7df8a6fafd31 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:55.633768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:55.649066 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96d964-78ee-499a-b32a-badb064a111e None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:55.651532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0682c-1936-4382-b086-7df8a6fafd31 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:55.675428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:55.684918 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eda3aa-d7ec-4a75-8f26-0351ab55f63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:55.686146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eda3aa-d7ec-4a75-8f26-0351ab55f63c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:55.686441 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eda3aa-d7ec-4a75-8f26-0351ab55f63c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:55.686706 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eda3aa-d7ec-4a75-8f26-0351ab55f63c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:55.689166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b069bd-2cc1-491a-8053-c8b1e266ae34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:55.690410 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b069bd-2cc1-491a-8053-c8b1e266ae34 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:55.697920 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b069bd-2cc1-491a-8053-c8b1e266ae34 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:55.697920 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b069bd-2cc1-491a-8053-c8b1e266ae34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:55.703249 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b069bd-2cc1-491a-8053-c8b1e266ae34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:55.704849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b069bd-2cc1-491a-8053-c8b1e266ae34 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:55.720581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-12b069bd-2cc1-491a-8053-c8b1e266ae34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:55.720581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:55.720581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:55.720581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:55.720581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:55.720581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:55.720581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:55.720581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:55.720581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:55.720581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:55.720581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:55.720581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:55.720581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:55.720581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:55.720581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:55.720581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:55.721792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:55.721792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:55.721792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:55.721792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:55.721792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:55.721792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:55.721792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:55.721792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:55.721792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:55.721792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:55.721792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:55.721792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:55.721792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:55.721792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:55.721792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:55.721792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:55.721792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:55.722860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:55.722860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:55.722860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:55.724156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 342/672] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:55.727240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b069bd-2cc1-491a-8053-c8b1e266ae34 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:55.763291 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eda3aa-d7ec-4a75-8f26-0351ab55f63c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:55.765022 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eda3aa-d7ec-4a75-8f26-0351ab55f63c None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:55.833815 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b069bd-2cc1-491a-8053-c8b1e266ae34 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:55.872930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4bf352-e05b-44a1-8e78-948c99d8da2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:55.873784 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bf352-e05b-44a1-8e78-948c99d8da2e None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:55.874161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bf352-e05b-44a1-8e78-948c99d8da2e None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:55.874530 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bf352-e05b-44a1-8e78-948c99d8da2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:55.877952 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:55.879494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 331/673] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:55.880602 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eda3aa-d7ec-4a75-8f26-0351ab55f63c None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:55.883661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4bf352-e05b-44a1-8e78-948c99d8da2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:55.886163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4bf352-e05b-44a1-8e78-948c99d8da2e None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:55.898087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-3b4bf352-e05b-44a1-8e78-948c99d8da2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:55.898087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:55.898087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:55.898087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:55.898087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:55.898087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:55.898087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:55.898087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:55.898087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:55.898087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:55.898087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:55.898087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:55.898087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:24:55.898087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:24:55.898087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:24:55.899411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:24:55.899411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:55.899411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:55.899411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:55.899411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:55.899411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:55.899411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:55.899411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:55.899411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:55.899411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:55.899411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:55.899411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:55.899411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:55.899411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:55.899411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:55.899411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:55.899411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:55.900545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:24:55.900545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:24:55.900545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:24:55.900545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:24:55.900545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:24:55.900545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:24:55.900545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:55.900545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:55.906873 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 343/674] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:55.906873 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4bf352-e05b-44a1-8e78-948c99d8da2e None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:55.934987 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:56.029763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eda3aa-d7ec-4a75-8f26-0351ab55f63c None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:56.031100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4bf352-e05b-44a1-8e78-948c99d8da2e None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:56.073636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16824057-5e5d-4814-8de6-788181bff1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:56.075053 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16824057-5e5d-4814-8de6-788181bff1da None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:56.075698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16824057-5e5d-4814-8de6-788181bff1da None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:56.075698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16824057-5e5d-4814-8de6-788181bff1da None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:56.076647 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16824057-5e5d-4814-8de6-788181bff1da None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:56.077105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16824057-5e5d-4814-8de6-788181bff1da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:56.078790 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16824057-5e5d-4814-8de6-788181bff1da None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:56.081472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296fab3-8078-41b2-b757-61c290cd58b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:56.082801 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296fab3-8078-41b2-b757-61c290cd58b0 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:56.083237 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296fab3-8078-41b2-b757-61c290cd58b0 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:56.083638 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296fab3-8078-41b2-b757-61c290cd58b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:56.092919 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 332/675] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:56.094105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16824057-5e5d-4814-8de6-788181bff1da None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:56.097395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296fab3-8078-41b2-b757-61c290cd58b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:56.099974 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296fab3-8078-41b2-b757-61c290cd58b0 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:56.121639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-6296fab3-8078-41b2-b757-61c290cd58b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:56.121639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:56.121639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:56.121639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:56.121639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:56.121639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:56.121639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:56.121639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:56.121639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:56.121639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:56.121639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:56.121639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:56.121639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:56.121639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:56.121639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:56.121639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:56.122835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:56.122835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:56.122835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:56.122835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:56.122835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:56.122835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:56.122835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:56.122835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:56.122835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:56.122835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:56.122835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:56.122835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:56.122835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:56.122835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:56.122835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:56.122835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:56.122835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:56.123910 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:56.123910 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:56.123910 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:56.126389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 344/676] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:56.127692 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296fab3-8078-41b2-b757-61c290cd58b0 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:56.201993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:56.205328 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:56.221957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16824057-5e5d-4814-8de6-788181bff1da None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:56.265125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8495c-32e4-413e-8ec5-5e9bd11f5446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:56.265952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8495c-32e4-413e-8ec5-5e9bd11f5446 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:56.266363 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8495c-32e4-413e-8ec5-5e9bd11f5446 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:56.266770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8495c-32e4-413e-8ec5-5e9bd11f5446 None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:56.267624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8495c-32e4-413e-8ec5-5e9bd11f5446 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:56.268035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8495c-32e4-413e-8ec5-5e9bd11f5446 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:56.268518 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296fab3-8078-41b2-b757-61c290cd58b0 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:56.269793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8495c-32e4-413e-8ec5-5e9bd11f5446 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:56.288357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 333/677] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:24:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:56.288357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8495c-32e4-413e-8ec5-5e9bd11f5446 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:56.316009 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d458e-e171-4a63-8409-fa1f35d5a1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:56.316677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d458e-e171-4a63-8409-fa1f35d5a1aa None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:56.316953 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d458e-e171-4a63-8409-fa1f35d5a1aa None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:56.317223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d458e-e171-4a63-8409-fa1f35d5a1aa None admin] PATH_INFO: `/v3/users/swift` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:56.325641 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d458e-e171-4a63-8409-fa1f35d5a1aa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:56.327086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d458e-e171-4a63-8409-fa1f35d5a1aa None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:56.339447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-d10d458e-e171-4a63-8409-fa1f35d5a1aa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:24:56.339447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:56.339447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:56.339447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:56.339447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:56.339447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:56.339447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:56.339447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:56.339447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:56.339447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:56.339447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:56.339447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:56.339447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:24:56.339447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:24:56.339447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:24:56.339447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:24:56.340971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:56.340971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:56.340971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:24:56.340971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:56.340971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:24:56.340971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:56.340971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:56.340971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:56.340971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:56.340971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:56.340971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:56.340971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:56.340971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:56.340971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:56.340971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:56.340971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:56.340971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:56.340971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:56.341967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:56.341967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:56.341967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:24:56.341967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:24:56.341967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:24:56.341967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:24:56.341967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:24:56.341967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:24:56.341967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:24:56.341967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:56.341967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 345/678] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:24:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:56.341967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d458e-e171-4a63-8409-fa1f35d5a1aa None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:56.447656 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8495c-32e4-413e-8ec5-5e9bd11f5446 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:56.458437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d458e-e171-4a63-8409-fa1f35d5a1aa None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:56.472503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:56.503486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f99de9-20c0-4b26-8e68-ca49915ccefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:56.504225 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f99de9-20c0-4b26-8e68-ca49915ccefd None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:56.504580 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f99de9-20c0-4b26-8e68-ca49915ccefd None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:56.504938 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f99de9-20c0-4b26-8e68-ca49915ccefd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:56.505738 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f99de9-20c0-4b26-8e68-ca49915ccefd None admin] RBAC: Adding query filter params (role.id=f91580fb2e5840bcb4f4054732e2e796, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=4db08aa2e4ef4478a3909a876754814c) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:56.506115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f99de9-20c0-4b26-8e68-ca49915ccefd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:56.508610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f99de9-20c0-4b26-8e68-ca49915ccefd None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:56.509091 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9a099-0f3f-4afe-a12f-9b71d55ac5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:56.509740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9a099-0f3f-4afe-a12f-9b71d55ac5c3 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:56.510015 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9a099-0f3f-4afe-a12f-9b71d55ac5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:56.510281 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9a099-0f3f-4afe-a12f-9b71d55ac5c3 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:56.510938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9a099-0f3f-4afe-a12f-9b71d55ac5c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:56.511231 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9a099-0f3f-4afe-a12f-9b71d55ac5c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:56.512652 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9a099-0f3f-4afe-a12f-9b71d55ac5c3 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:56.516692 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 346/679] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:24:56 2022] GET /identity/v3/role_assignments?user.id=4db08aa2e4ef4478a3909a876754814c&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=f91580fb2e5840bcb4f4054732e2e796 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:56.517766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f99de9-20c0-4b26-8e68-ca49915ccefd None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:56.535161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 334/680] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:56.542682 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a51536-f3db-4b4e-8a5b-0621547ababc None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:56.542999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a51536-f3db-4b4e-8a5b-0621547ababc None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:56.543245 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a51536-f3db-4b4e-8a5b-0621547ababc None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:56.545031 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 335/681] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:56.545645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a51536-f3db-4b4e-8a5b-0621547ababc None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:56.564648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:56.658584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f99de9-20c0-4b26-8e68-ca49915ccefd None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:56.688056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a51536-f3db-4b4e-8a5b-0621547ababc None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:56.705286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276af7bc-c34d-4e4a-8611-a7c4bce971dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:56.705912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276af7bc-c34d-4e4a-8611-a7c4bce971dc None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:56.706201 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276af7bc-c34d-4e4a-8611-a7c4bce971dc None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:56.706473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276af7bc-c34d-4e4a-8611-a7c4bce971dc None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:56.707215 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276af7bc-c34d-4e4a-8611-a7c4bce971dc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:56.707853 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276af7bc-c34d-4e4a-8611-a7c4bce971dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:56.709732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276af7bc-c34d-4e4a-8611-a7c4bce971dc None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:56.732497 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9afc9-231a-4490-87ed-dde9789b761a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:56.733276 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9afc9-231a-4490-87ed-dde9789b761a None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:56.733650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9afc9-231a-4490-87ed-dde9789b761a None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:56.734021 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9afc9-231a-4490-87ed-dde9789b761a None admin] PATH_INFO: `/v3/projects/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:56.735303 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 347/682] 10.222.0.59 () {56 vars in 1135 bytes} [Mon Aug 29 22:24:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:56.740886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a434895-e5cc-48b8-a955-662ebb197012 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:56.741194 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a434895-e5cc-48b8-a955-662ebb197012 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:56.741472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a434895-e5cc-48b8-a955-662ebb197012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:56.744672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b9afc9-231a-4490-87ed-dde9789b761a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:56.746268 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b9afc9-231a-4490-87ed-dde9789b761a None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:56.760884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-58b9afc9-231a-4490-87ed-dde9789b761a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:56.760884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:56.760884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:56.760884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:56.760884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:56.760884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:56.760884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:56.760884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:56.760884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:56.760884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:56.760884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:56.760884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:56.760884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:24:56.760884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:24:56.760884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:24:56.762027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:24:56.762027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:56.762027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:56.762027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:56.762027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:56.762027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:56.762027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:56.762027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:56.762027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:56.762027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:56.762027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:56.762027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:56.762027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:56.762027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:56.762027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:56.762027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:56.762027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:56.763125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:24:56.763125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:24:56.763125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:24:56.763125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:24:56.763125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:24:56.763125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:24:56.763125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:56.763125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:56.764101 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 336/683] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:56.765039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-7a434895-e5cc-48b8-a955-662ebb197012 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:56.765167 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9afc9-231a-4490-87ed-dde9789b761a None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:56.766367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-7a434895-e5cc-48b8-a955-662ebb197012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:56.801412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:56.849628 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:56.868811 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 348/684] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:56.870228 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a434895-e5cc-48b8-a955-662ebb197012 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:56.926366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9afc9-231a-4490-87ed-dde9789b761a None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:56.998595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a434895-e5cc-48b8-a955-662ebb197012 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:57.019470 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a184e3-4a69-4657-b0f9-49e082dedca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:57.021177 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a184e3-4a69-4657-b0f9-49e082dedca5 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:57.021759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a184e3-4a69-4657-b0f9-49e082dedca5 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:57.023363 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a184e3-4a69-4657-b0f9-49e082dedca5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:57.065283 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e040cf-bdff-42c3-8309-c5f69e82ecff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:57.066812 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e040cf-bdff-42c3-8309-c5f69e82ecff None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:57.067394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e040cf-bdff-42c3-8309-c5f69e82ecff None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:57.067914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e040cf-bdff-42c3-8309-c5f69e82ecff None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:57.069145 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e040cf-bdff-42c3-8309-c5f69e82ecff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:57.069604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e040cf-bdff-42c3-8309-c5f69e82ecff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:57.072735 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e040cf-bdff-42c3-8309-c5f69e82ecff None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:57.090163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 349/685] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:24:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:57.092257 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d67fad-09a4-4f96-ba41-55bdc3f11193 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:57.092991 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d67fad-09a4-4f96-ba41-55bdc3f11193 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:57.093411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d67fad-09a4-4f96-ba41-55bdc3f11193 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:57.095323 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 350/686] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:57.096654 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d67fad-09a4-4f96-ba41-55bdc3f11193 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:57.129680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a184e3-4a69-4657-b0f9-49e082dedca5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:57.131130 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a184e3-4a69-4657-b0f9-49e082dedca5 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:57.131352 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:57.206076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:57.243715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d67fad-09a4-4f96-ba41-55bdc3f11193 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:57.245546 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 337/687] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:57.246718 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cc7f0-65df-4983-98ea-6a5fe0ac470b None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:57.250547 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cc7f0-65df-4983-98ea-6a5fe0ac470b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:57.250547 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cc7f0-65df-4983-98ea-6a5fe0ac470b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:57.266606 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-9f9cc7f0-65df-4983-98ea-6a5fe0ac470b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:57.267814 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-9f9cc7f0-65df-4983-98ea-6a5fe0ac470b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:57.288083 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f59cb-1076-4cfd-ae51-be8fac312775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:57.288725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f59cb-1076-4cfd-ae51-be8fac312775 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:57.289033 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f59cb-1076-4cfd-ae51-be8fac312775 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:57.289452 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f59cb-1076-4cfd-ae51-be8fac312775 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:57.290480 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f59cb-1076-4cfd-ae51-be8fac312775 None admin] RBAC: Adding query filter params (role.id=f91580fb2e5840bcb4f4054732e2e796, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=cb802d01e71b486cb40c6388430d674d) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:57.290852 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f59cb-1076-4cfd-ae51-be8fac312775 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:57.292307 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f59cb-1076-4cfd-ae51-be8fac312775 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:57.308098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 351/688] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:24:57 2022] GET /identity/v3/role_assignments?user.id=cb802d01e71b486cb40c6388430d674d&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=f91580fb2e5840bcb4f4054732e2e796 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:57.308098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f59cb-1076-4cfd-ae51-be8fac312775 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:57.365291 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 338/689] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:57.366367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cc7f0-65df-4983-98ea-6a5fe0ac470b None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:57.474749 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:57.499087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f59cb-1076-4cfd-ae51-be8fac312775 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:57.559267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff47ab-b94d-497f-8522-ca689ab3d0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:57.560477 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff47ab-b94d-497f-8522-ca689ab3d0d8 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:57.561349 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff47ab-b94d-497f-8522-ca689ab3d0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:57.561828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff47ab-b94d-497f-8522-ca689ab3d0d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:57.568306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:57.577510 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cc7f0-65df-4983-98ea-6a5fe0ac470b None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:57.592233 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff47ab-b94d-497f-8522-ca689ab3d0d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:24:57.599831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-29ff47ab-b94d-497f-8522-ca689ab3d0d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:24:57.599831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:57.599831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:57.599831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:57.599831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:57.599831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:57.599831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:57.599831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:57.599831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:57.599831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:57.599831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:57.599831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:57.599831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:24:57.599831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:24:57.599831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:24:57.599831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:24:57.600973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:24:57.600973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:24:57.600973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:24:57.600973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:24:57.600973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:57.600973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:57.600973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:57.600973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:57.600973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:57.600973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:57.600973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:57.600973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:57.600973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:57.600973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:57.600973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:57.600973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:57.602627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:57.602627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:57.602627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:57.602627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:57.602627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:24:57.602627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:24:57.602627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:24:57.602627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:24:57.602627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:24:57.602627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:24:57.602627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:24:57.602627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:57.602627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 352/690] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:24:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:57.609573 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff47ab-b94d-497f-8522-ca689ab3d0d8 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:57.639428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077bdfd-adc9-4bb2-be11-fd18f993dd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:57.640073 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077bdfd-adc9-4bb2-be11-fd18f993dd7e None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:57.640403 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077bdfd-adc9-4bb2-be11-fd18f993dd7e None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:57.640670 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077bdfd-adc9-4bb2-be11-fd18f993dd7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:57.653757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077bdfd-adc9-4bb2-be11-fd18f993dd7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:57.655212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077bdfd-adc9-4bb2-be11-fd18f993dd7e None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:57.675960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-c077bdfd-adc9-4bb2-be11-fd18f993dd7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:57.675960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:57.675960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:57.675960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:57.675960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:57.675960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:57.675960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:57.675960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:57.675960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:57.675960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:57.675960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:57.675960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:57.675960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:57.675960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:57.675960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:57.675960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:57.678035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:57.678035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:57.678035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:57.678035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:57.678035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:57.678035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:57.678035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:57.678035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:57.678035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:57.678035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:57.678035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:57.678035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:57.678035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:57.678035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:57.678035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:57.678035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:57.678035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:57.685454 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:57.685454 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:57.685454 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:57.685454 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 339/691] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:57.688798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077bdfd-adc9-4bb2-be11-fd18f993dd7e None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:57.788260 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff47ab-b94d-497f-8522-ca689ab3d0d8 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:57.838755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e606c1-3f44-4806-9f05-125869289918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:57.839066 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077bdfd-adc9-4bb2-be11-fd18f993dd7e None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:57.839282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e606c1-3f44-4806-9f05-125869289918 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:57.839556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e606c1-3f44-4806-9f05-125869289918 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:57.839823 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e606c1-3f44-4806-9f05-125869289918 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:57.840520 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e606c1-3f44-4806-9f05-125869289918 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:57.840812 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e606c1-3f44-4806-9f05-125869289918 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:57.843626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e606c1-3f44-4806-9f05-125869289918 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:57.855336 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 353/692] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:24:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:57.856307 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e606c1-3f44-4806-9f05-125869289918 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:57.869080 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:57.889229 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212090e-5da9-4409-9478-78bdcf9b5317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:57.889861 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212090e-5da9-4409-9478-78bdcf9b5317 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:57.890135 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212090e-5da9-4409-9478-78bdcf9b5317 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:57.890402 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212090e-5da9-4409-9478-78bdcf9b5317 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:57.891059 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212090e-5da9-4409-9478-78bdcf9b5317 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:57.891358 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212090e-5da9-4409-9478-78bdcf9b5317 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:57.892787 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212090e-5da9-4409-9478-78bdcf9b5317 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:57.910080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 340/693] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:24:57.911938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212090e-5da9-4409-9478-78bdcf9b5317 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:57.934633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:58.020377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e606c1-3f44-4806-9f05-125869289918 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:58.044964 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212090e-5da9-4409-9478-78bdcf9b5317 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:58.076045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f959e6-4bb0-4942-bfb8-5db18c49fc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:58.078250 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f959e6-4bb0-4942-bfb8-5db18c49fc3f None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:58.079362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f959e6-4bb0-4942-bfb8-5db18c49fc3f None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:58.079877 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f959e6-4bb0-4942-bfb8-5db18c49fc3f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:58.080965 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f959e6-4bb0-4942-bfb8-5db18c49fc3f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:58.082877 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f959e6-4bb0-4942-bfb8-5db18c49fc3f None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:58.084251 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b98a27-accc-40c8-8245-ec3d1e759825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:58.084987 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b98a27-accc-40c8-8245-ec3d1e759825 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:58.085262 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b98a27-accc-40c8-8245-ec3d1e759825 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:58.086394 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b98a27-accc-40c8-8245-ec3d1e759825 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:58.098382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b98a27-accc-40c8-8245-ec3d1e759825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:58.099462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-27f959e6-4bb0-4942-bfb8-5db18c49fc3f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 22:24:58.099462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:58.099462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:24:58.099462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:24:58.099462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:24:58.099462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:24:58.099462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:24:58.099462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:24:58.099462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 22:24:58.099462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:58.099462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:24:58.099462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:58.099462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:58.099462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:58.099462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:58.099462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:58.099462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:58.100472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:58.100472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:58.100472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:58.100472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:58.100472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:58.100472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:58.100472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:24:58.100472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:24:58.100472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:24:58.100472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:24:58.100472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:58.100472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:58.100472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:58.100472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:58.100472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:58.100472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:58.101912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:58.101912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:58.101912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:58.101912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:58.101912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:58.101912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:58.101912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:58.101912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:58.101912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:58.101912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:58.101912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:24:58.101912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:24:58.101912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 22:24:58.101912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:58.102743 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b98a27-accc-40c8-8245-ec3d1e759825 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:58.104683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 354/694] 10.222.0.59 () {56 vars in 1129 bytes} [Mon Aug 29 22:24:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:58.105758 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f959e6-4bb0-4942-bfb8-5db18c49fc3f None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:58.129907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-14b98a27-accc-40c8-8245-ec3d1e759825 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:58.129907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:58.129907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:58.129907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:58.129907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:58.129907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:58.129907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:58.129907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:58.129907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:58.129907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:58.129907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:58.129907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:58.129907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:58.129907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:58.129907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:58.129907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:58.131106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:58.131106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:58.131106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:58.131106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:58.131106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:58.131106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:58.131106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:58.131106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:58.131106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:58.131106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:58.131106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:58.131106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:58.131106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:58.131106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:58.131106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:58.131106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:58.131106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:58.132348 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:58.132348 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:58.132348 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:58.134718 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 341/695] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:58.135732 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b98a27-accc-40c8-8245-ec3d1e759825 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:58.213112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:58.236630 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:58.253086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f959e6-4bb0-4942-bfb8-5db18c49fc3f None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:58.297942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b98a27-accc-40c8-8245-ec3d1e759825 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:58.302185 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141709e-a35f-40a1-a34a-fe43a92d4de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:58.303284 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141709e-a35f-40a1-a34a-fe43a92d4de4 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:58.303734 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141709e-a35f-40a1-a34a-fe43a92d4de4 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:58.304094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141709e-a35f-40a1-a34a-fe43a92d4de4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:58.315356 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141709e-a35f-40a1-a34a-fe43a92d4de4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:58.317024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141709e-a35f-40a1-a34a-fe43a92d4de4 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:58.330898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-d141709e-a35f-40a1-a34a-fe43a92d4de4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:58.330898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:58.330898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:58.330898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:58.330898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:58.330898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:58.330898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:58.330898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:58.330898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:58.330898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:58.330898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:58.330898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:58.330898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:24:58.330898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:24:58.330898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:24:58.332169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:24:58.332169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:58.332169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:58.332169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:58.332169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:58.332169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:58.332169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:58.332169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:58.332169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:58.332169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:58.332169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:58.332169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:58.332169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:58.332169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:58.332169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:58.332169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:58.332169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:58.333819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:24:58.333819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:24:58.333819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:24:58.333819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:24:58.333819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:24:58.333819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:24:58.333819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:24:58.333819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:58.335097 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 355/696] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:24:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:58.336661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141709e-a35f-40a1-a34a-fe43a92d4de4 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:58.354919 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6629a-4ef1-4ad6-b756-3ddc0a65f6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:58.356025 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6629a-4ef1-4ad6-b756-3ddc0a65f6c7 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:58.356499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6629a-4ef1-4ad6-b756-3ddc0a65f6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:58.357327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6629a-4ef1-4ad6-b756-3ddc0a65f6c7 None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:58.358219 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6629a-4ef1-4ad6-b756-3ddc0a65f6c7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:58.358781 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6629a-4ef1-4ad6-b756-3ddc0a65f6c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:58.360399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6629a-4ef1-4ad6-b756-3ddc0a65f6c7 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:58.369000 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 342/697] 10.222.0.59 () {56 vars in 1124 bytes} [Mon Aug 29 22:24:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:58.370844 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6629a-4ef1-4ad6-b756-3ddc0a65f6c7 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:58.477057 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141709e-a35f-40a1-a34a-fe43a92d4de4 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:58.526659 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:58.534924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1cd6ab-193b-4e14-88c3-07d797681293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:58.543707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cd6ab-193b-4e14-88c3-07d797681293 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:58.543707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cd6ab-193b-4e14-88c3-07d797681293 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:58.543707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cd6ab-193b-4e14-88c3-07d797681293 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:58.543707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1cd6ab-193b-4e14-88c3-07d797681293 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:58.543707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1cd6ab-193b-4e14-88c3-07d797681293 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:58.543707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1cd6ab-193b-4e14-88c3-07d797681293 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:58.550039 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6629a-4ef1-4ad6-b756-3ddc0a65f6c7 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:58.556791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 356/698] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:58.558398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1cd6ab-193b-4e14-88c3-07d797681293 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:58.587901 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:58.604170 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28411d08-4e39-47c2-a2e4-f95ab59ee24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:58.604828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28411d08-4e39-47c2-a2e4-f95ab59ee24b None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:58.605184 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28411d08-4e39-47c2-a2e4-f95ab59ee24b None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:58.605400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28411d08-4e39-47c2-a2e4-f95ab59ee24b None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:58.607067 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28411d08-4e39-47c2-a2e4-f95ab59ee24b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:58.607372 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28411d08-4e39-47c2-a2e4-f95ab59ee24b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:58.608966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28411d08-4e39-47c2-a2e4-f95ab59ee24b None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:58.627323 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 343/699] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:24:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:58.628295 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28411d08-4e39-47c2-a2e4-f95ab59ee24b None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:58.710052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1cd6ab-193b-4e14-88c3-07d797681293 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:58.755020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269990eb-ddc0-41d9-a8b9-45d543171c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:58.755679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269990eb-ddc0-41d9-a8b9-45d543171c8b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:58.755957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269990eb-ddc0-41d9-a8b9-45d543171c8b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:58.756248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269990eb-ddc0-41d9-a8b9-45d543171c8b None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:58.756887 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269990eb-ddc0-41d9-a8b9-45d543171c8b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:58.757185 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269990eb-ddc0-41d9-a8b9-45d543171c8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:58.759075 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269990eb-ddc0-41d9-a8b9-45d543171c8b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:58.772181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 357/700] 10.222.0.59 () {56 vars in 1124 bytes} [Mon Aug 29 22:24:58 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:58.772181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269990eb-ddc0-41d9-a8b9-45d543171c8b None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:58.827194 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28411d08-4e39-47c2-a2e4-f95ab59ee24b None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:58.860545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:58.910426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d495641-4eaf-48c2-b35c-c331a4ffe719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:58.911474 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d495641-4eaf-48c2-b35c-c331a4ffe719 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:58.911934 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d495641-4eaf-48c2-b35c-c331a4ffe719 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:58.912445 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d495641-4eaf-48c2-b35c-c331a4ffe719 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:58.935685 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d495641-4eaf-48c2-b35c-c331a4ffe719 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:58.937760 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d495641-4eaf-48c2-b35c-c331a4ffe719 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:58.955672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-3d495641-4eaf-48c2-b35c-c331a4ffe719 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:24:58.955672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:58.955672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:58.955672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:58.955672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:58.955672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:58.955672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:58.955672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:58.955672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:58.955672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:58.955672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:58.955672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:58.955672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:24:58.955672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:24:58.955672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:24:58.955672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:24:58.957549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:58.957549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:58.957549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:24:58.957549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:58.957549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:24:58.957549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:24:58.957549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:58.957549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:58.957549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:58.957549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:58.957549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:58.957549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:58.957549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:58.957549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:58.957549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:58.957549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:58.957549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:58.957549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:58.958593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:58.958593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:58.958593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:24:58.958593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:24:58.958593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:24:58.958593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:24:58.958593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:24:58.958593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:24:58.958593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:24:58.958593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:58.958593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 344/701] 10.222.0.59 () {56 vars in 1093 bytes} [Mon Aug 29 22:24:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:58.958593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f330f0b-eb0f-4b21-99b3-b20de858f102 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:58.959224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f330f0b-eb0f-4b21-99b3-b20de858f102 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:58.959224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f330f0b-eb0f-4b21-99b3-b20de858f102 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:58.965002 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 345/702] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:58.965002 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f330f0b-eb0f-4b21-99b3-b20de858f102 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:58.986508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269990eb-ddc0-41d9-a8b9-45d543171c8b None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:59.038085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b5313-3886-41c1-b918-7775839f01ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:59.039746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b5313-3886-41c1-b918-7775839f01ac None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:59.039746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b5313-3886-41c1-b918-7775839f01ac None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:59.040029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b5313-3886-41c1-b918-7775839f01ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:59.040751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b5313-3886-41c1-b918-7775839f01ac None admin] RBAC: Adding query filter params (role.id=f91580fb2e5840bcb4f4054732e2e796, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=f1f0503ae1b24ce6a8adb3cfd7973f14) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:59.041753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b5313-3886-41c1-b918-7775839f01ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:59.043855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b5313-3886-41c1-b918-7775839f01ac None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:59.044733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:59.049705 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 358/703] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:24:58 2022] GET /identity/v3/role_assignments?user.id=f1f0503ae1b24ce6a8adb3cfd7973f14&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=f91580fb2e5840bcb4f4054732e2e796 => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:59.050937 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad1825-2e82-408b-b38e-b2e97e45cc99 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:59.051216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad1825-2e82-408b-b38e-b2e97e45cc99 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:59.051483 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad1825-2e82-408b-b38e-b2e97e45cc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:59.072859 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-1cad1825-2e82-408b-b38e-b2e97e45cc99 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:24:59.074186 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-1cad1825-2e82-408b-b38e-b2e97e45cc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:24:59.185645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f330f0b-eb0f-4b21-99b3-b20de858f102 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:59.208489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 359/704] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:24:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:24:59.240357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e4521-ea3a-4bf1-b79e-aaf9bc8b53cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:59.241446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e4521-ea3a-4bf1-b79e-aaf9bc8b53cb None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:59.241752 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e4521-ea3a-4bf1-b79e-aaf9bc8b53cb None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:59.242051 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e4521-ea3a-4bf1-b79e-aaf9bc8b53cb None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:59.242802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e4521-ea3a-4bf1-b79e-aaf9bc8b53cb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:24:59.243360 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e4521-ea3a-4bf1-b79e-aaf9bc8b53cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:59.245267 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e4521-ea3a-4bf1-b79e-aaf9bc8b53cb None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:59.265571 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 346/705] 10.222.0.59 () {56 vars in 1133 bytes} [Mon Aug 29 22:24:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:24:59.274480 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad1825-2e82-408b-b38e-b2e97e45cc99 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:59.345848 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:59.408802 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad1825-2e82-408b-b38e-b2e97e45cc99 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:59.448122 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400671ec-4e49-439b-8504-ee195ccb41ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:59.448675 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400671ec-4e49-439b-8504-ee195ccb41ad None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:59.448948 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400671ec-4e49-439b-8504-ee195ccb41ad None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:59.449208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400671ec-4e49-439b-8504-ee195ccb41ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:59.523233 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400671ec-4e49-439b-8504-ee195ccb41ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:59.524951 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400671ec-4e49-439b-8504-ee195ccb41ad None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:59.525958 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e4521-ea3a-4bf1-b79e-aaf9bc8b53cb None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:59.588245 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:59.663240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 360/706] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:24:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:24:59.665667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e4521-ea3a-4bf1-b79e-aaf9bc8b53cb None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:59.666273 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:59.674651 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400671ec-4e49-439b-8504-ee195ccb41ad None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:59.713583 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b5e5f-a953-4370-81a3-8f1bccdbc99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:59.714534 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b5e5f-a953-4370-81a3-8f1bccdbc99c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:59.715007 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b5e5f-a953-4370-81a3-8f1bccdbc99c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:59.715468 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b5e5f-a953-4370-81a3-8f1bccdbc99c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:59.729544 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b5e5f-a953-4370-81a3-8f1bccdbc99c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:59.731310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b5e5f-a953-4370-81a3-8f1bccdbc99c None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:59.759856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-3b0b5e5f-a953-4370-81a3-8f1bccdbc99c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:59.759856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:59.759856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:59.759856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:59.759856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:59.759856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:59.759856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:59.759856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:59.759856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:59.759856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:59.759856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:59.759856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:59.759856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:59.759856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:59.759856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:59.759856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:59.760935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:59.760935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:59.760935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:59.760935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:59.760935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:59.760935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:59.760935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:59.760935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:59.760935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:59.760935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:59.760935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:59.760935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:59.760935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:59.760935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:59.760935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:59.760935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:59.760935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:59.761813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:59.761813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:59.761813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:24:59.775550 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 347/707] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:59.784176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b5e5f-a953-4370-81a3-8f1bccdbc99c None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:59.869976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400671ec-4e49-439b-8504-ee195ccb41ad None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:59.923268 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f7b564-11ee-402c-957c-220ef449de84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:59.923268 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7b564-11ee-402c-957c-220ef449de84 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:59.923268 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7b564-11ee-402c-957c-220ef449de84 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:59.923268 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f7b564-11ee-402c-957c-220ef449de84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:59.937322 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7b564-11ee-402c-957c-220ef449de84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:24:59.939036 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f7b564-11ee-402c-957c-220ef449de84 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:24:59.946468 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b5e5f-a953-4370-81a3-8f1bccdbc99c None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:24:59.951196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:24:59.960973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-77f7b564-11ee-402c-957c-220ef449de84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:59.960973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:24:59.960973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:24:59.960973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:24:59.960973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:24:59.960973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:24:59.960973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:24:59.960973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:24:59.960973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:24:59.960973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:24:59.960973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:24:59.960973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:24:59.960973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:24:59.960973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:24:59.960973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:24:59.960973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:24:59.967400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:24:59.967400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:24:59.967400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:24:59.967400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:24:59.967400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:24:59.967400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:24:59.967400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:24:59.967400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:24:59.967400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:24:59.967400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:24:59.967400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:24:59.967400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:24:59.967400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:24:59.967400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:24:59.967400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:24:59.967400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:24:59.967400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:24:59.968662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:24:59.968662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:24:59.968662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:24:59.968662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 361/708] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:24:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:24:59.968662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83680dbd-4783-4e30-b36c-02af28342661 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:59.969318 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83680dbd-4783-4e30-b36c-02af28342661 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:59.969585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83680dbd-4783-4e30-b36c-02af28342661 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:59.975025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 362/709] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:24:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:24:59.975025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83680dbd-4783-4e30-b36c-02af28342661 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:24:59.997170 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d17bf-4f38-4104-b9bd-581e3dee0211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:24:59.998136 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d17bf-4f38-4104-b9bd-581e3dee0211 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:24:59.998615 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d17bf-4f38-4104-b9bd-581e3dee0211 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:24:59.999088 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d17bf-4f38-4104-b9bd-581e3dee0211 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:24:59.999951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d17bf-4f38-4104-b9bd-581e3dee0211 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:00.000487 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d17bf-4f38-4104-b9bd-581e3dee0211 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:00.002052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d17bf-4f38-4104-b9bd-581e3dee0211 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:00.020730 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 348/710] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:00.022178 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53796225-5f59-4277-b319-a75a51e10ed9 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:00.022459 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53796225-5f59-4277-b319-a75a51e10ed9 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:00.022724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53796225-5f59-4277-b319-a75a51e10ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:00.046369 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-53796225-5f59-4277-b319-a75a51e10ed9 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:00.047676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-53796225-5f59-4277-b319-a75a51e10ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:00.061891 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:00.114777 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83680dbd-4783-4e30-b36c-02af28342661 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:00.142882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 349/711] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:25:00.142882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53796225-5f59-4277-b319-a75a51e10ed9 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:00.168638 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacbaa4d-a3f0-4942-af71-7eda8d6d5de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:00.169246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbaa4d-a3f0-4942-af71-7eda8d6d5de7 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:00.169518 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbaa4d-a3f0-4942-af71-7eda8d6d5de7 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:00.169782 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbaa4d-a3f0-4942-af71-7eda8d6d5de7 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:00.170428 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacbaa4d-a3f0-4942-af71-7eda8d6d5de7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:00.170723 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacbaa4d-a3f0-4942-af71-7eda8d6d5de7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:00.172077 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacbaa4d-a3f0-4942-af71-7eda8d6d5de7 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:00.200531 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 363/712] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:00.206106 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacbaa4d-a3f0-4942-af71-7eda8d6d5de7 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:00.328673 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:00.347094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53796225-5f59-4277-b319-a75a51e10ed9 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:00.392127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd993d4f-605f-4495-b8ab-06c5bf312e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:00.392803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd993d4f-605f-4495-b8ab-06c5bf312e88 None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:00.393132 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd993d4f-605f-4495-b8ab-06c5bf312e88 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:00.393341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd993d4f-605f-4495-b8ab-06c5bf312e88 None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:00.393979 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd993d4f-605f-4495-b8ab-06c5bf312e88 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:00.395387 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd993d4f-605f-4495-b8ab-06c5bf312e88 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:00.405373 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacbaa4d-a3f0-4942-af71-7eda8d6d5de7 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:00.441194 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c437b-0bff-4664-9251-35b2f0cfc6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:00.442178 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c437b-0bff-4664-9251-35b2f0cfc6fc None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:00.442614 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c437b-0bff-4664-9251-35b2f0cfc6fc None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:00.443039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c437b-0bff-4664-9251-35b2f0cfc6fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:00.450897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c437b-0bff-4664-9251-35b2f0cfc6fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:00.452644 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c437b-0bff-4664-9251-35b2f0cfc6fc None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:00.465883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-dc4c437b-0bff-4664-9251-35b2f0cfc6fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:00.465883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:00.465883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:00.465883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:00.465883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:00.465883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:00.465883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:00.465883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:00.465883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:00.465883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:00.465883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:00.465883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:00.465883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:25:00.465883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:25:00.465883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:25:00.466849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:25:00.466849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:00.466849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:00.466849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:00.466849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:00.466849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:00.466849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:00.466849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:00.466849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:00.466849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:00.466849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:00.466849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:00.466849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:00.466849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:00.466849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:00.466849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:00.466849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:00.467766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:25:00.467766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:25:00.467766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:25:00.467766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:25:00.467766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:25:00.467766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:25:00.467766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:00.467766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:00.470240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 364/713] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:00.472842 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c437b-0bff-4664-9251-35b2f0cfc6fc None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:00.536822 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:00.548578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 350/714] 10.222.0.59 () {60 vars in 1132 bytes} [Mon Aug 29 22:25:00 2022] POST /identity/v3/users => generated 254 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:25:00.550002 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd993d4f-605f-4495-b8ab-06c5bf312e88 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:00.594592 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c437b-0bff-4664-9251-35b2f0cfc6fc None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:00.622197 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:00.640940 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4a355-93d7-418c-b68b-13b01b68cb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:00.641995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4a355-93d7-418c-b68b-13b01b68cb93 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:00.642497 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4a355-93d7-418c-b68b-13b01b68cb93 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:00.642978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4a355-93d7-418c-b68b-13b01b68cb93 None admin] PATH_INFO: `/v3/services/network` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:00.643960 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4a355-93d7-418c-b68b-13b01b68cb93 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:00.646104 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4a355-93d7-418c-b68b-13b01b68cb93 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:00.657528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-b3e4a355-93d7-418c-b68b-13b01b68cb93 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 22:25:00.657528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:00.657528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:00.657528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:00.657528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:00.657528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:00.657528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:00.657528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:00.657528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 22:25:00.657528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:00.657528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:00.657528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:00.657528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:00.657528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:00.657528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:00.657528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:00.657528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:00.658646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:00.658646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:00.658646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:00.658646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:00.658646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:00.658646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:00.658646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:00.658646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:00.658646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:00.658646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:00.658646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:00.658646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:00.658646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:00.658646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:00.658646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:00.658646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:00.659545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:00.659545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:00.659545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:00.659545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:00.659545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:00.659545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:00.659545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:00.659545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:00.659545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:00.659545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:00.659545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:00.659545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:00.659545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 22:25:00.659545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:00.663013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 365/715] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:00 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:00.670947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4a355-93d7-418c-b68b-13b01b68cb93 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:00.774765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd993d4f-605f-4495-b8ab-06c5bf312e88 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:00.818354 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c9bbd-ee82-43c1-b7dd-96372945536f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:00.818968 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c9bbd-ee82-43c1-b7dd-96372945536f None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:00.819240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c9bbd-ee82-43c1-b7dd-96372945536f None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:00.819506 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c9bbd-ee82-43c1-b7dd-96372945536f None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:00.821176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c9bbd-ee82-43c1-b7dd-96372945536f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:00.821547 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c9bbd-ee82-43c1-b7dd-96372945536f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:00.823258 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c9bbd-ee82-43c1-b7dd-96372945536f None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:00.848271 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 351/716] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:25:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:00.850933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c9bbd-ee82-43c1-b7dd-96372945536f None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:00.853858 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4a355-93d7-418c-b68b-13b01b68cb93 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:00.894548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7cb556-a6ea-4195-88bd-9ece151a4f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:00.895180 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cb556-a6ea-4195-88bd-9ece151a4f95 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:00.895402 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cb556-a6ea-4195-88bd-9ece151a4f95 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:00.895690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cb556-a6ea-4195-88bd-9ece151a4f95 None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:00.897267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7cb556-a6ea-4195-88bd-9ece151a4f95 None admin] RBAC: Adding query filter params (name=network) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:00.897597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7cb556-a6ea-4195-88bd-9ece151a4f95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:00.899508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7cb556-a6ea-4195-88bd-9ece151a4f95 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:00.908269 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 366/717] 10.222.0.59 () {56 vars in 1112 bytes} [Mon Aug 29 22:25:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:00.913356 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7cb556-a6ea-4195-88bd-9ece151a4f95 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:00.940604 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:00.984975 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c9bbd-ee82-43c1-b7dd-96372945536f None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:01.024252 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ca509-868f-4307-bb8e-0088759418d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:01.024884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ca509-868f-4307-bb8e-0088759418d3 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:01.025206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ca509-868f-4307-bb8e-0088759418d3 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:01.025528 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ca509-868f-4307-bb8e-0088759418d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:01.026294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ca509-868f-4307-bb8e-0088759418d3 None admin] RBAC: Adding query filter params (role.id=a836108447dc439e994611bc74d53d9c, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=7f21105679f245a19ea4850e6c010dbf) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:01.026601 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ca509-868f-4307-bb8e-0088759418d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:01.028078 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ca509-868f-4307-bb8e-0088759418d3 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:01.034205 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 352/718] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:25:00 2022] GET /identity/v3/role_assignments?user.id=7f21105679f245a19ea4850e6c010dbf&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=a836108447dc439e994611bc74d53d9c => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:01.044654 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:01.099405 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7cb556-a6ea-4195-88bd-9ece151a4f95 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:01.158991 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df171d3-628f-4b22-b99f-4221f6314dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:01.159622 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df171d3-628f-4b22-b99f-4221f6314dcb None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:01.159901 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df171d3-628f-4b22-b99f-4221f6314dcb None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:01.160192 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df171d3-628f-4b22-b99f-4221f6314dcb None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:01.160845 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df171d3-628f-4b22-b99f-4221f6314dcb None admin] RBAC: Adding query filter params (type=network) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:01.161146 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df171d3-628f-4b22-b99f-4221f6314dcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:01.162530 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df171d3-628f-4b22-b99f-4221f6314dcb None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:01.169181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 367/719] 10.222.0.59 () {56 vars in 1112 bytes} [Mon Aug 29 22:25:00 2022] GET /identity/v3/services?type=network => generated 124 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:01.183620 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e62e2-fc91-4038-ad38-a4cdbaf5fc53 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:01.183940 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e62e2-fc91-4038-ad38-a4cdbaf5fc53 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:01.184171 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e62e2-fc91-4038-ad38-a4cdbaf5fc53 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:01.185688 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 353/720] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:01.194751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb53fa8-3844-4281-bd16-bd0a478fcb75 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:01.200356 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb53fa8-3844-4281-bd16-bd0a478fcb75 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:01.200356 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb53fa8-3844-4281-bd16-bd0a478fcb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:01.227991 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-0bb53fa8-3844-4281-bd16-bd0a478fcb75 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:01.229286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-0bb53fa8-3844-4281-bd16-bd0a478fcb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:01.308350 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 368/721] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:25:01.646853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e62e2-fc91-4038-ad38-a4cdbaf5fc53 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:01.701862 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34d026-72f5-411f-a187-742a6ac185f4 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:01.702190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34d026-72f5-411f-a187-742a6ac185f4 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:01.702417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34d026-72f5-411f-a187-742a6ac185f4 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:01.703879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 369/722] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:25:01.709348 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e086450-f814-4106-a57a-d72bc969d528 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:01.709879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e086450-f814-4106-a57a-d72bc969d528 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:01.710941 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e086450-f814-4106-a57a-d72bc969d528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:01.730919 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-5e086450-f814-4106-a57a-d72bc969d528 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:01.732473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-5e086450-f814-4106-a57a-d72bc969d528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:01.784116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e62e2-fc91-4038-ad38-a4cdbaf5fc53 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:01.820892 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:01.826211 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3488eb8-5beb-4e56-84e1-9ef4a2d026c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:01.826850 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3488eb8-5beb-4e56-84e1-9ef4a2d026c4 None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:01.827130 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3488eb8-5beb-4e56-84e1-9ef4a2d026c4 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:01.827404 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3488eb8-5beb-4e56-84e1-9ef4a2d026c4 None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:01.828080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3488eb8-5beb-4e56-84e1-9ef4a2d026c4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:01.829499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3488eb8-5beb-4e56-84e1-9ef4a2d026c4 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:01.838607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:01.874178 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 370/723] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 22:25:01.942075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 354/724] 10.222.0.59 () {60 vars in 1141 bytes} [Mon Aug 29 22:25:01 2022] POST /identity/v3/services => generated 251 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:25:02.208339 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e086450-f814-4106-a57a-d72bc969d528 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:02.330792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e086450-f814-4106-a57a-d72bc969d528 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:02.377604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9bd77-7674-4c11-a626-724790ef04ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:02.378614 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9bd77-7674-4c11-a626-724790ef04ba None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:02.379100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9bd77-7674-4c11-a626-724790ef04ba None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:02.379561 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9bd77-7674-4c11-a626-724790ef04ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:02.391256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9bd77-7674-4c11-a626-724790ef04ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:25:02.396289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-e8b9bd77-7674-4c11-a626-724790ef04ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:25:02.396289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:02.396289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:02.396289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:02.396289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:02.396289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:02.396289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:02.396289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:02.396289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:02.396289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:02.396289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:02.396289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:02.396289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:25:02.396289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:25:02.396289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:25:02.396289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:25:02.397198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:25:02.397198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:25:02.397198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:25:02.397198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:25:02.397198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:02.397198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:02.397198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:02.397198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:02.397198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:02.397198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:02.397198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:02.397198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:02.397198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:02.397198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:02.397198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:02.397198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:02.398053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:02.398053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:02.398053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:02.398053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:02.398053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:25:02.398053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:25:02.398053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:25:02.398053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:25:02.398053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:25:02.398053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:25:02.398053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:25:02.398053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:02.400882 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 371/725] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:25:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:02.406928 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3488eb8-5beb-4e56-84e1-9ef4a2d026c4 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:02.547791 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3488eb8-5beb-4e56-84e1-9ef4a2d026c4 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:02.592896 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0f841-10ad-4016-8468-c53dcc35b93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:02.593528 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0f841-10ad-4016-8468-c53dcc35b93c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:02.593913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0f841-10ad-4016-8468-c53dcc35b93c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:02.594191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0f841-10ad-4016-8468-c53dcc35b93c None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:02.594990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0f841-10ad-4016-8468-c53dcc35b93c None admin] RBAC: Adding query filter params (name=admin) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:02.595296 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0f841-10ad-4016-8468-c53dcc35b93c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:02.596696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0f841-10ad-4016-8468-c53dcc35b93c None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:02.607039 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 355/726] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:25:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:02.612659 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9bd77-7674-4c11-a626-724790ef04ba None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:02.667813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:02.776352 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9bd77-7674-4c11-a626-724790ef04ba None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:02.831257 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931ce05-73a8-4298-a110-b9aaf333f7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:02.831813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931ce05-73a8-4298-a110-b9aaf333f7d4 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:02.832085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931ce05-73a8-4298-a110-b9aaf333f7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:02.832375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931ce05-73a8-4298-a110-b9aaf333f7d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:02.857747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4931ce05-73a8-4298-a110-b9aaf333f7d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:02.860263 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4931ce05-73a8-4298-a110-b9aaf333f7d4 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:02.890605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-4931ce05-73a8-4298-a110-b9aaf333f7d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:02.890605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:02.890605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:02.890605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:02.890605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:02.890605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:02.890605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:02.890605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:02.890605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:02.890605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:02.890605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:02.890605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:02.890605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:02.890605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:02.890605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:02.890605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:02.891653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:02.891653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:02.891653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:02.891653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:02.891653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:02.891653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:02.891653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:02.891653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:02.891653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:02.891653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:02.891653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:02.891653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:02.891653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:02.891653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:02.891653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:02.891653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:02.891653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:02.892632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:02.892632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:02.892632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:02.892800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 372/727] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:02.899852 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0f841-10ad-4016-8468-c53dcc35b93c None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:02.972370 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:03.050317 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0f841-10ad-4016-8468-c53dcc35b93c None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:03.095242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd4d91-52e4-46be-a4c1-1d11c36b65fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:03.095999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd4d91-52e4-46be-a4c1-1d11c36b65fd None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:03.096502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd4d91-52e4-46be-a4c1-1d11c36b65fd None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:03.096956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd4d91-52e4-46be-a4c1-1d11c36b65fd None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:03.097626 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd4d91-52e4-46be-a4c1-1d11c36b65fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:03.097933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd4d91-52e4-46be-a4c1-1d11c36b65fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:03.099306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd4d91-52e4-46be-a4c1-1d11c36b65fd None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:03.117696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 356/728] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:03.125390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931ce05-73a8-4298-a110-b9aaf333f7d4 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:03.270593 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:03.300060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931ce05-73a8-4298-a110-b9aaf333f7d4 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:03.350147 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f386a-0a18-4b5a-a6be-d3ed22f0406a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:03.350706 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f386a-0a18-4b5a-a6be-d3ed22f0406a None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:03.351562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f386a-0a18-4b5a-a6be-d3ed22f0406a None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:03.351854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f386a-0a18-4b5a-a6be-d3ed22f0406a None admin] PATH_INFO: `/v3/users/swift` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:03.362423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f386a-0a18-4b5a-a6be-d3ed22f0406a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:03.363866 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f386a-0a18-4b5a-a6be-d3ed22f0406a None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:03.378634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-cc0f386a-0a18-4b5a-a6be-d3ed22f0406a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:25:03.378634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:03.378634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:03.378634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:03.378634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:03.378634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:03.378634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:03.378634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:03.378634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:03.378634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:03.378634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:03.378634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:03.378634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:25:03.378634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:25:03.378634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:25:03.378634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:25:03.379624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:03.379624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:03.379624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:25:03.379624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:03.379624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:25:03.379624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:03.379624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:03.379624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:03.379624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:03.379624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:03.379624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:03.379624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:03.379624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:03.379624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:03.379624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:03.379624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:03.379624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:03.379624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:03.380577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:03.380577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:03.380577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:25:03.380577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:25:03.380577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:25:03.380577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:25:03.380577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:25:03.380577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:25:03.380577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:25:03.380577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:03.380577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 373/729] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:25:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:03.387908 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd4d91-52e4-46be-a4c1-1d11c36b65fd None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:03.538054 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd4d91-52e4-46be-a4c1-1d11c36b65fd None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:03.547560 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:03.583406 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4b109-7ca2-4a1c-ba1d-e7c477259aed None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:03.583834 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4b109-7ca2-4a1c-ba1d-e7c477259aed None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:03.584308 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4b109-7ca2-4a1c-ba1d-e7c477259aed None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:03.586636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 374/730] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:03.592170 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582a49f-8521-4762-bf48-f14d34f8e822 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:03.592730 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582a49f-8521-4762-bf48-f14d34f8e822 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:03.593218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582a49f-8521-4762-bf48-f14d34f8e822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:03.608027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7359a35-4d3a-4be2-bd10-07518ee0f8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:03.608027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7359a35-4d3a-4be2-bd10-07518ee0f8cb None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:03.608027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7359a35-4d3a-4be2-bd10-07518ee0f8cb None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:03.608027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7359a35-4d3a-4be2-bd10-07518ee0f8cb None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:03.608417 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7359a35-4d3a-4be2-bd10-07518ee0f8cb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:03.608417 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7359a35-4d3a-4be2-bd10-07518ee0f8cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:03.608417 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7359a35-4d3a-4be2-bd10-07518ee0f8cb None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:03.628089 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-c582a49f-8521-4762-bf48-f14d34f8e822 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:03.630437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c582a49f-8521-4762-bf48-f14d34f8e822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:03.646078 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 357/731] 10.222.0.59 () {56 vars in 1135 bytes} [Mon Aug 29 22:25:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:03.647006 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae06b83-6f05-40d0-8c08-8d01cc2b37b5 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:03.647663 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae06b83-6f05-40d0-8c08-8d01cc2b37b5 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:03.648529 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae06b83-6f05-40d0-8c08-8d01cc2b37b5 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:03.650346 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 358/732] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:03.658880 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac18ee-384e-41fa-9216-bb1c12e46c0e None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:03.659160 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac18ee-384e-41fa-9216-bb1c12e46c0e None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:03.659829 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac18ee-384e-41fa-9216-bb1c12e46c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:03.682600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-bfac18ee-384e-41fa-9216-bb1c12e46c0e None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:03.684116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-bfac18ee-384e-41fa-9216-bb1c12e46c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:03.725916 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 375/733] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:25:03.727221 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582a49f-8521-4762-bf48-f14d34f8e822 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:03.770418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 359/734] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:25:03.772636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f8633-5ed9-4947-a20e-cfe86dbeec61 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:03.772961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f8633-5ed9-4947-a20e-cfe86dbeec61 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:03.773259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f8633-5ed9-4947-a20e-cfe86dbeec61 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:03.778456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 360/735] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:03.780513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce3c6a-5e38-434e-a2ea-a5826547502a None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:03.780847 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce3c6a-5e38-434e-a2ea-a5826547502a None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:03.781118 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce3c6a-5e38-434e-a2ea-a5826547502a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:03.805895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-acce3c6a-5e38-434e-a2ea-a5826547502a None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:03.807433 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-acce3c6a-5e38-434e-a2ea-a5826547502a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:03.826388 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:03.850060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582a49f-8521-4762-bf48-f14d34f8e822 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:03.891671 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cee016-13be-4a1f-9f4f-1d9eeda6d776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:03.892703 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cee016-13be-4a1f-9f4f-1d9eeda6d776 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:03.893195 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cee016-13be-4a1f-9f4f-1d9eeda6d776 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:03.893656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cee016-13be-4a1f-9f4f-1d9eeda6d776 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:03.900158 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 361/736] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 22:25:04.006026 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cee016-13be-4a1f-9f4f-1d9eeda6d776 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:04.008614 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cee016-13be-4a1f-9f4f-1d9eeda6d776 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:04.019529 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce3c6a-5e38-434e-a2ea-a5826547502a None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:04.069507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:04.118317 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:04.157606 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 376/737] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:25:03 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 22:25:04.158530 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cee016-13be-4a1f-9f4f-1d9eeda6d776 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:04.188651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce3c6a-5e38-434e-a2ea-a5826547502a None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:04.233646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885306c-a241-4970-a580-6daba8cb963e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:04.234859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885306c-a241-4970-a580-6daba8cb963e None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:04.235137 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885306c-a241-4970-a580-6daba8cb963e None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:04.235444 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885306c-a241-4970-a580-6daba8cb963e None admin] PATH_INFO: `/v3/roles/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:04.250808 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885306c-a241-4970-a580-6daba8cb963e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:25:04.258047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-6885306c-a241-4970-a580-6daba8cb963e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:25:04.258047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:04.258047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:04.258047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:04.258047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:04.258047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:04.258047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:04.258047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:04.258047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:04.258047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:04.258047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:04.258047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:04.258047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:25:04.258047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:25:04.258047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:25:04.258047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:25:04.259953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:25:04.259953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:25:04.259953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:25:04.259953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:25:04.259953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:04.259953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:04.259953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:04.259953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:04.259953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:04.259953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:04.259953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:04.259953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:04.259953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:04.259953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:04.259953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:04.259953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:04.262144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:04.262144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:04.262144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:04.262144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:04.262144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:25:04.262144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:25:04.262144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:25:04.262144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:25:04.262144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:25:04.262144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:25:04.262144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:25:04.262144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:04.262144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 362/738] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:25:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:04.262144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885306c-a241-4970-a580-6daba8cb963e None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:04.275419 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cee016-13be-4a1f-9f4f-1d9eeda6d776 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:04.314424 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2dbc3-23e4-4e5c-ba1a-0bbf79fef6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:04.315168 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2dbc3-23e4-4e5c-ba1a-0bbf79fef6bd None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:04.315762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2dbc3-23e4-4e5c-ba1a-0bbf79fef6bd None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:04.315762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2dbc3-23e4-4e5c-ba1a-0bbf79fef6bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:04.327095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2dbc3-23e4-4e5c-ba1a-0bbf79fef6bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:25:04.331930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-a0c2dbc3-23e4-4e5c-ba1a-0bbf79fef6bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:25:04.331930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:04.331930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:04.331930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:04.331930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:04.331930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:04.331930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:04.331930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:04.331930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:04.331930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:04.331930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:04.331930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:04.331930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:25:04.331930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:25:04.331930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:25:04.331930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:25:04.334556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:25:04.334556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:25:04.334556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:25:04.334556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:25:04.334556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:04.334556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:04.334556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:04.334556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:04.334556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:04.334556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:04.334556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:04.334556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:04.334556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:04.334556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:04.334556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:04.334556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:04.338060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:04.338060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:04.338060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:04.338060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:04.338060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:25:04.338060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:25:04.338060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:25:04.338060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:25:04.338060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:25:04.338060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:25:04.338060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:25:04.338060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:04.339863 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 377/739] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:25:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:04.340273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2dbc3-23e4-4e5c-ba1a-0bbf79fef6bd None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:04.362809 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:04.375730 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:04.380023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885306c-a241-4970-a580-6daba8cb963e None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:04.416194 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13881904-0d2a-4225-ab43-08c9a9a7b742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:04.417076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13881904-0d2a-4225-ab43-08c9a9a7b742 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:04.417439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13881904-0d2a-4225-ab43-08c9a9a7b742 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:04.417610 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13881904-0d2a-4225-ab43-08c9a9a7b742 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:04.428760 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13881904-0d2a-4225-ab43-08c9a9a7b742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:04.430733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13881904-0d2a-4225-ab43-08c9a9a7b742 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:04.447076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-13881904-0d2a-4225-ab43-08c9a9a7b742 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:04.447076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:04.447076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:04.447076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:04.447076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:04.447076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:04.447076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:04.447076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:04.447076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:04.447076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:04.447076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:04.447076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:04.447076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:04.447076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:04.447076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:04.447076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:04.448068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:04.448068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:04.448068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:04.448068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:04.448068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:04.448068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:04.448068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:04.448068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:04.448068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:04.448068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:04.448068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:04.448068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:04.448068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:04.448068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:04.448068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:04.448068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:04.448068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:04.449090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:04.449090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:04.449090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:04.449090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 363/740] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:04.449727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13881904-0d2a-4225-ab43-08c9a9a7b742 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:04.456413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2dbc3-23e4-4e5c-ba1a-0bbf79fef6bd None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:04.501838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dea113-1e66-4a61-a526-e59faf0a5b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:04.502443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dea113-1e66-4a61-a526-e59faf0a5b20 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:04.502719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dea113-1e66-4a61-a526-e59faf0a5b20 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:04.502981 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dea113-1e66-4a61-a526-e59faf0a5b20 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:04.503637 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dea113-1e66-4a61-a526-e59faf0a5b20 None admin] RBAC: Adding query filter params (name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:04.503934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dea113-1e66-4a61-a526-e59faf0a5b20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:04.505334 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dea113-1e66-4a61-a526-e59faf0a5b20 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:04.514535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 378/741] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:25:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:04.515373 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dea113-1e66-4a61-a526-e59faf0a5b20 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:04.569434 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13881904-0d2a-4225-ab43-08c9a9a7b742 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:04.606636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c227a8-3ef0-4455-8121-053193ac45ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:04.607390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c227a8-3ef0-4455-8121-053193ac45ab None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:04.607568 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c227a8-3ef0-4455-8121-053193ac45ab None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:04.607800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c227a8-3ef0-4455-8121-053193ac45ab None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:04.608480 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c227a8-3ef0-4455-8121-053193ac45ab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:04.609841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c227a8-3ef0-4455-8121-053193ac45ab None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:04.615121 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:04.622541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:04.630883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dea113-1e66-4a61-a526-e59faf0a5b20 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:04.665275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d676c-4c0d-4983-80ce-6b005a861872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:04.665873 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d676c-4c0d-4983-80ce-6b005a861872 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:04.666151 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d676c-4c0d-4983-80ce-6b005a861872 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:04.666468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d676c-4c0d-4983-80ce-6b005a861872 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:04.667220 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d676c-4c0d-4983-80ce-6b005a861872 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:04.667519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d676c-4c0d-4983-80ce-6b005a861872 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:04.668905 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d676c-4c0d-4983-80ce-6b005a861872 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:04.677773 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 379/742] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:25:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:04.678705 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d676c-4c0d-4983-80ce-6b005a861872 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:04.687770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 364/743] 10.222.0.59 () {60 vars in 1141 bytes} [Mon Aug 29 22:25:04 2022] POST /identity/v3/services => generated 240 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:25:04.688597 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c227a8-3ef0-4455-8121-053193ac45ab None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:04.798345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d676c-4c0d-4983-80ce-6b005a861872 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:04.822161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c227a8-3ef0-4455-8121-053193ac45ab None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:04.842401 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa7b76-e780-4cfe-98f4-3c6c6d6488f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:04.843494 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa7b76-e780-4cfe-98f4-3c6c6d6488f3 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:04.843930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa7b76-e780-4cfe-98f4-3c6c6d6488f3 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:04.844246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa7b76-e780-4cfe-98f4-3c6c6d6488f3 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:04.844884 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa7b76-e780-4cfe-98f4-3c6c6d6488f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:04.845184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa7b76-e780-4cfe-98f4-3c6c6d6488f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:04.846559 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa7b76-e780-4cfe-98f4-3c6c6d6488f3 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:04.859634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 380/744] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:04.861471 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4baa5-e53a-4ab4-96ee-51091a368142 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:04.861761 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4baa5-e53a-4ab4-96ee-51091a368142 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:04.862032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4baa5-e53a-4ab4-96ee-51091a368142 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:04.863430 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 381/745] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:04.867702 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4baa5-e53a-4ab4-96ee-51091a368142 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:04.870699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d2a39-3074-42dd-ad39-a1d4d3a3e597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:04.871348 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d2a39-3074-42dd-ad39-a1d4d3a3e597 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:04.871623 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d2a39-3074-42dd-ad39-a1d4d3a3e597 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:04.871889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d2a39-3074-42dd-ad39-a1d4d3a3e597 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:04.882668 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:04.884856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d2a39-3074-42dd-ad39-a1d4d3a3e597 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:04.886329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d2a39-3074-42dd-ad39-a1d4d3a3e597 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:04.905926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-454d2a39-3074-42dd-ad39-a1d4d3a3e597 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:04.905926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:04.905926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:04.905926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:04.905926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:04.905926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:04.905926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:04.905926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:04.905926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:04.905926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:04.905926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:04.905926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:04.905926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:04.905926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:04.905926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:04.905926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:04.907476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:04.907476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:04.907476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:04.907476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:04.907476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:04.907476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:04.907476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:04.907476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:04.907476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:04.907476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:04.907476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:04.907476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:04.907476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:04.907476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:04.907476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:04.907476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:04.907476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:04.908407 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:04.908407 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:04.908407 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:04.908407 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 365/746] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:04.909368 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d2a39-3074-42dd-ad39-a1d4d3a3e597 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:04.962478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:04.995614 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4baa5-e53a-4ab4-96ee-51091a368142 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:05.022022 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d2a39-3074-42dd-ad39-a1d4d3a3e597 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:05.032538 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61033b-df4c-46f5-a7a2-d683b475fc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:05.033173 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61033b-df4c-46f5-a7a2-d683b475fc79 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:05.033452 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61033b-df4c-46f5-a7a2-d683b475fc79 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:05.033724 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61033b-df4c-46f5-a7a2-d683b475fc79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:05.045344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61033b-df4c-46f5-a7a2-d683b475fc79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:05.046783 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61033b-df4c-46f5-a7a2-d683b475fc79 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:05.059649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb503139-1b0e-4a76-a9eb-a497fae867ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:05.060319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb503139-1b0e-4a76-a9eb-a497fae867ba None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:05.060597 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb503139-1b0e-4a76-a9eb-a497fae867ba None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:05.060895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb503139-1b0e-4a76-a9eb-a497fae867ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:05.061922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-4d61033b-df4c-46f5-a7a2-d683b475fc79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:05.061922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:05.061922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:05.061922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:05.061922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:05.061922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:05.061922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:05.061922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:05.061922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:05.061922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:05.061922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:05.061922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:05.061922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:05.061922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:05.061922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:05.061922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:05.062856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:05.062856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:05.062856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:05.062856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:05.062856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:05.062856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:05.062856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:05.062856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:05.062856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:05.062856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:05.062856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:05.062856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:05.062856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:05.062856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:05.062856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:05.062856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:05.062856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:05.063755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:05.063755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:05.063755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:05.063755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 382/747] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:05.064556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176badde-717d-4cd9-920f-16948848d453 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:05.064556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176badde-717d-4cd9-920f-16948848d453 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:05.064809 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176badde-717d-4cd9-920f-16948848d453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:05.073939 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb503139-1b0e-4a76-a9eb-a497fae867ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:05.075536 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb503139-1b0e-4a76-a9eb-a497fae867ba None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:05.086270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-176badde-717d-4cd9-920f-16948848d453 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:05.087469 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-176badde-717d-4cd9-920f-16948848d453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:05.087920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-bb503139-1b0e-4a76-a9eb-a497fae867ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:05.087920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:05.087920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:05.087920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:05.087920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:05.087920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:05.087920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:05.087920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:05.087920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:05.087920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:05.087920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:05.087920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:05.087920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:25:05.087920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:25:05.087920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:25:05.088845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:25:05.088845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:05.088845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:05.088845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:05.088845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:05.088845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:05.088845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:05.088845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:05.088845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:05.088845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:05.088845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:05.088845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:05.088845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:05.088845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:05.088845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:05.088845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:05.088845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:05.089727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:25:05.089727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:25:05.089727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:25:05.089727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:25:05.089727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:25:05.089727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:25:05.089727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:05.089727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:05.093995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 366/748] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:05.095324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb503139-1b0e-4a76-a9eb-a497fae867ba None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:05.157217 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:05.166216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 383/749] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:05.174279 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176badde-717d-4cd9-920f-16948848d453 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:05.209412 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb503139-1b0e-4a76-a9eb-a497fae867ba None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:05.214004 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:05.251458 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ffa0c-a36f-4869-adc5-631fe0428992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:05.252074 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ffa0c-a36f-4869-adc5-631fe0428992 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:05.252381 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ffa0c-a36f-4869-adc5-631fe0428992 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:05.252724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ffa0c-a36f-4869-adc5-631fe0428992 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:05.253443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ffa0c-a36f-4869-adc5-631fe0428992 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:05.253788 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ffa0c-a36f-4869-adc5-631fe0428992 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:05.255349 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ffa0c-a36f-4869-adc5-631fe0428992 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:05.270916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 367/750] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:05.277492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ffa0c-a36f-4869-adc5-631fe0428992 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:05.308335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176badde-717d-4cd9-920f-16948848d453 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:05.355396 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8a63a-ad8e-4df0-94d6-901954c93245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:05.356006 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8a63a-ad8e-4df0-94d6-901954c93245 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:05.356332 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8a63a-ad8e-4df0-94d6-901954c93245 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:05.356605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8a63a-ad8e-4df0-94d6-901954c93245 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:05.357250 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8a63a-ad8e-4df0-94d6-901954c93245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:05.357564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8a63a-ad8e-4df0-94d6-901954c93245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:05.358956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8a63a-ad8e-4df0-94d6-901954c93245 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:05.379256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 384/751] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:05.379256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8a63a-ad8e-4df0-94d6-901954c93245 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:05.403426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ffa0c-a36f-4869-adc5-631fe0428992 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:05.444038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78305891-9443-4b7d-aabe-09d958e1372f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:05.444312 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78305891-9443-4b7d-aabe-09d958e1372f None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:05.444612 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78305891-9443-4b7d-aabe-09d958e1372f None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:05.444882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78305891-9443-4b7d-aabe-09d958e1372f None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:05.445585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78305891-9443-4b7d-aabe-09d958e1372f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:05.445889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78305891-9443-4b7d-aabe-09d958e1372f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:05.447302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78305891-9443-4b7d-aabe-09d958e1372f None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:05.455817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:05.459373 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 368/752] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:25:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:05.461699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78305891-9443-4b7d-aabe-09d958e1372f None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:05.494483 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:05.511893 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8a63a-ad8e-4df0-94d6-901954c93245 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:05.546860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24b032-172e-4277-b4f7-99816538368f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:05.547510 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24b032-172e-4277-b4f7-99816538368f None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:05.547791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24b032-172e-4277-b4f7-99816538368f None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:05.548059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24b032-172e-4277-b4f7-99816538368f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:05.557300 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24b032-172e-4277-b4f7-99816538368f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:05.559140 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24b032-172e-4277-b4f7-99816538368f None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:05.570904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-8c24b032-172e-4277-b4f7-99816538368f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:25:05.570904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:05.570904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:05.570904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:05.570904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:05.570904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:05.570904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:05.570904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:05.570904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:05.570904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:05.570904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:05.570904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:05.570904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:25:05.570904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:25:05.570904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:25:05.570904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:25:05.571914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:05.571914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:05.571914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:25:05.571914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:05.571914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:25:05.571914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:05.571914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:05.571914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:05.571914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:05.571914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:05.571914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:05.571914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:05.571914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:05.571914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:05.571914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:05.571914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:05.571914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:05.571914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:05.576762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:05.576762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:05.576762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:25:05.576762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:25:05.576762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:25:05.576762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:25:05.576762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:25:05.576762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:25:05.576762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:25:05.576762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:05.576762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 385/753] 10.222.0.59 () {56 vars in 1117 bytes} [Mon Aug 29 22:25:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:05.576762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24b032-172e-4277-b4f7-99816538368f None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:05.589738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78305891-9443-4b7d-aabe-09d958e1372f None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:05.629240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408cb872-ecea-4591-ad42-ca7782bdab49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:05.629860 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408cb872-ecea-4591-ad42-ca7782bdab49 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:05.630137 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408cb872-ecea-4591-ad42-ca7782bdab49 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:05.630404 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408cb872-ecea-4591-ad42-ca7782bdab49 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:05.638268 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408cb872-ecea-4591-ad42-ca7782bdab49 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:05.639694 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408cb872-ecea-4591-ad42-ca7782bdab49 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:05.650990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-408cb872-ecea-4591-ad42-ca7782bdab49 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:25:05.650990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:05.650990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:05.650990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:05.650990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:05.650990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:05.650990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:05.650990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:05.650990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:05.650990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:05.650990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:05.650990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:05.650990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:25:05.650990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:25:05.650990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:25:05.650990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:25:05.652071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:05.652071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:05.652071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:25:05.652071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:05.652071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:25:05.652071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:05.652071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:05.652071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:05.652071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:05.652071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:05.652071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:05.652071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:05.652071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:05.652071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:05.652071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:05.652071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:05.652071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:05.652071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:05.653047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:05.653047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:05.653047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:25:05.653047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:25:05.653047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:25:05.653047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:25:05.653047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:25:05.653047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:25:05.653047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:25:05.653047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:05.653047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 369/754] 10.222.0.59 () {56 vars in 1093 bytes} [Mon Aug 29 22:25:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:05.653641 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408cb872-ecea-4591-ad42-ca7782bdab49 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:05.695907 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24b032-172e-4277-b4f7-99816538368f None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:05.723603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:05.743991 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b7cdf-3bd0-4b4c-b697-ac4edc6b41f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:05.744655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b7cdf-3bd0-4b4c-b697-ac4edc6b41f4 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:05.754416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b7cdf-3bd0-4b4c-b697-ac4edc6b41f4 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:05.754416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b7cdf-3bd0-4b4c-b697-ac4edc6b41f4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:05.754416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b7cdf-3bd0-4b4c-b697-ac4edc6b41f4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:05.754416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b7cdf-3bd0-4b4c-b697-ac4edc6b41f4 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:05.754704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:05.756577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-a13b7cdf-3bd0-4b4c-b697-ac4edc6b41f4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 22:25:05.756577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:05.756577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:05.756577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:05.756577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:05.756577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:05.756577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:05.756577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:05.756577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 22:25:05.756577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:05.756577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:05.756577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:05.756577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:05.756577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:05.756577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:05.756577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:05.756577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:05.757543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:05.757543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:05.757543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:05.757543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:05.757543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:05.757543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:05.757543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:05.757543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:05.757543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:05.757543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:05.757543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:05.757543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:05.757543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:05.757543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:05.757543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:05.757543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:05.758457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:05.758457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:05.758457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:05.758457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:05.758457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:05.758457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:05.758457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:05.758457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:05.758457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:05.758457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:05.758457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:05.758457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:05.758457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 22:25:05.758457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:05.758457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 386/755] 10.222.0.59 () {56 vars in 1129 bytes} [Mon Aug 29 22:25:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:05.759253 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b7cdf-3bd0-4b4c-b697-ac4edc6b41f4 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:05.770849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408cb872-ecea-4591-ad42-ca7782bdab49 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:05.811718 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be745614-4f8e-4b1e-9ace-cd41ff976068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:05.812489 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be745614-4f8e-4b1e-9ace-cd41ff976068 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:05.812853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be745614-4f8e-4b1e-9ace-cd41ff976068 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:05.813191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be745614-4f8e-4b1e-9ace-cd41ff976068 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:05.814010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be745614-4f8e-4b1e-9ace-cd41ff976068 None admin] RBAC: Adding query filter params (role.id=a836108447dc439e994611bc74d53d9c, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=f1f0503ae1b24ce6a8adb3cfd7973f14) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:05.814381 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be745614-4f8e-4b1e-9ace-cd41ff976068 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:05.815933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be745614-4f8e-4b1e-9ace-cd41ff976068 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:05.822732 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 370/756] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:25:05 2022] GET /identity/v3/role_assignments?user.id=f1f0503ae1b24ce6a8adb3cfd7973f14&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=a836108447dc439e994611bc74d53d9c => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:05.823429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be745614-4f8e-4b1e-9ace-cd41ff976068 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:05.902498 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b7cdf-3bd0-4b4c-b697-ac4edc6b41f4 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:05.950578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be745614-4f8e-4b1e-9ace-cd41ff976068 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:05.957108 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f0caf-92b3-4b7c-91a5-5a762843ba3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:05.958107 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f0caf-92b3-4b7c-91a5-5a762843ba3b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:05.959218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f0caf-92b3-4b7c-91a5-5a762843ba3b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:05.959703 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f0caf-92b3-4b7c-91a5-5a762843ba3b None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:05.961496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f0caf-92b3-4b7c-91a5-5a762843ba3b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:05.962035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f0caf-92b3-4b7c-91a5-5a762843ba3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:05.964439 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f0caf-92b3-4b7c-91a5-5a762843ba3b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:05.984816 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 387/757] 10.222.0.59 () {56 vars in 1149 bytes} [Mon Aug 29 22:25:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:05.985671 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f0caf-92b3-4b7c-91a5-5a762843ba3b None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:06.001111 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee424a1-f9c7-496e-ae48-ff0034fb7871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:06.002151 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee424a1-f9c7-496e-ae48-ff0034fb7871 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:06.002692 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee424a1-f9c7-496e-ae48-ff0034fb7871 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:06.002973 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee424a1-f9c7-496e-ae48-ff0034fb7871 None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:06.003708 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee424a1-f9c7-496e-ae48-ff0034fb7871 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:06.004042 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee424a1-f9c7-496e-ae48-ff0034fb7871 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:06.006304 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee424a1-f9c7-496e-ae48-ff0034fb7871 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:06.037124 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 371/758] 10.222.0.59 () {56 vars in 1133 bytes} [Mon Aug 29 22:25:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:06.037325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:06.038112 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee424a1-f9c7-496e-ae48-ff0034fb7871 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:06.052524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:06.124397 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f0caf-92b3-4b7c-91a5-5a762843ba3b None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:06.171349 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c38767-aa05-4cbb-8548-ef3c65c6602f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:06.172611 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c38767-aa05-4cbb-8548-ef3c65c6602f None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:06.173195 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c38767-aa05-4cbb-8548-ef3c65c6602f None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:06.173538 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c38767-aa05-4cbb-8548-ef3c65c6602f None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:06.174299 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c38767-aa05-4cbb-8548-ef3c65c6602f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:06.175104 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c38767-aa05-4cbb-8548-ef3c65c6602f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:06.178094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c38767-aa05-4cbb-8548-ef3c65c6602f None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:06.180312 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee424a1-f9c7-496e-ae48-ff0034fb7871 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:06.191017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 388/759] 10.222.0.59 () {56 vars in 1124 bytes} [Mon Aug 29 22:25:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:06.191017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c38767-aa05-4cbb-8548-ef3c65c6602f None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:06.234920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c0ba6-b230-422e-a6f4-c12b0818bdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:06.235606 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c0ba6-b230-422e-a6f4-c12b0818bdff None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:06.235945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c0ba6-b230-422e-a6f4-c12b0818bdff None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:06.236042 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c0ba6-b230-422e-a6f4-c12b0818bdff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:06.332907 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c38767-aa05-4cbb-8548-ef3c65c6602f None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:06.336828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:06.350289 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c0ba6-b230-422e-a6f4-c12b0818bdff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:06.358360 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c0ba6-b230-422e-a6f4-c12b0818bdff None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:06.364979 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:06.390746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4524e309-4a0f-4c82-b64b-c57382887f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:06.391765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524e309-4a0f-4c82-b64b-c57382887f44 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:06.392290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524e309-4a0f-4c82-b64b-c57382887f44 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:06.393045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524e309-4a0f-4c82-b64b-c57382887f44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:06.405984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4524e309-4a0f-4c82-b64b-c57382887f44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:06.408165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4524e309-4a0f-4c82-b64b-c57382887f44 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:06.425404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-4524e309-4a0f-4c82-b64b-c57382887f44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:06.425404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:06.425404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:06.425404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:06.425404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:06.425404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:06.425404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:06.425404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:06.425404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:06.425404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:06.425404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:06.425404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:06.425404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:06.425404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:06.425404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:06.425404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:06.426412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:06.426412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:06.426412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:06.426412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:06.426412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:06.426412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:06.426412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:06.426412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:06.426412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:06.426412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:06.426412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:06.426412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:06.426412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:06.426412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:06.426412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:06.426412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:06.426412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:06.427298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:06.427298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:06.427298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:06.430244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 389/760] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:06.431391 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4524e309-4a0f-4c82-b64b-c57382887f44 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:06.489662 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 372/761] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:25:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:25:06.490694 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c0ba6-b230-422e-a6f4-c12b0818bdff None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:06.547050 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4524e309-4a0f-4c82-b64b-c57382887f44 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:06.584127 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecff399-15ff-4a1e-8a41-f360f72e2f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:06.585205 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecff399-15ff-4a1e-8a41-f360f72e2f1b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:06.585877 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecff399-15ff-4a1e-8a41-f360f72e2f1b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:06.586357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecff399-15ff-4a1e-8a41-f360f72e2f1b None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:06.587249 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecff399-15ff-4a1e-8a41-f360f72e2f1b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:06.587763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecff399-15ff-4a1e-8a41-f360f72e2f1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:06.589641 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecff399-15ff-4a1e-8a41-f360f72e2f1b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:06.601313 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 390/762] 10.222.0.59 () {56 vars in 1124 bytes} [Mon Aug 29 22:25:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:06.602600 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecff399-15ff-4a1e-8a41-f360f72e2f1b None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:06.620434 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c0ba6-b230-422e-a6f4-c12b0818bdff None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:06.625471 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:06.632482 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:06.662960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c822e-053f-4a26-9b69-4f760ce36c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:06.664005 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c822e-053f-4a26-9b69-4f760ce36c11 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:06.664655 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c822e-053f-4a26-9b69-4f760ce36c11 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:06.665115 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c822e-053f-4a26-9b69-4f760ce36c11 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:06.666248 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c822e-053f-4a26-9b69-4f760ce36c11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:06.666746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c822e-053f-4a26-9b69-4f760ce36c11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:06.668618 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c822e-053f-4a26-9b69-4f760ce36c11 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:06.690206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 373/763] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:06.691509 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c822e-053f-4a26-9b69-4f760ce36c11 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:06.726569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecff399-15ff-4a1e-8a41-f360f72e2f1b None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:06.766799 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e75533-23aa-4270-9d8b-697957d185ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:06.767665 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e75533-23aa-4270-9d8b-697957d185ec None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:06.768093 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e75533-23aa-4270-9d8b-697957d185ec None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:06.768586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e75533-23aa-4270-9d8b-697957d185ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:06.781922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e75533-23aa-4270-9d8b-697957d185ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:06.783732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e75533-23aa-4270-9d8b-697957d185ec None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:06.800880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-f1e75533-23aa-4270-9d8b-697957d185ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:06.800880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:06.800880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:06.800880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:06.800880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:06.800880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:06.800880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:06.800880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:06.800880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:06.800880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:06.800880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:06.800880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:06.800880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:06.800880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:06.800880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:06.800880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:06.801997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:06.801997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:06.801997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:06.801997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:06.801997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:06.801997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:06.801997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:06.801997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:06.801997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:06.801997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:06.801997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:06.801997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:06.801997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:06.801997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:06.801997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:06.801997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:06.801997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:06.808060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:06.808060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:06.808060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:06.808060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 391/764] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:06.808060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e75533-23aa-4270-9d8b-697957d185ec None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:06.836865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c822e-053f-4a26-9b69-4f760ce36c11 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:06.875002 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac2135-2bdb-468e-8a96-329bebb568cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:06.875613 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac2135-2bdb-468e-8a96-329bebb568cd None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:06.875889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac2135-2bdb-468e-8a96-329bebb568cd None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:06.876172 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac2135-2bdb-468e-8a96-329bebb568cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:06.876955 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac2135-2bdb-468e-8a96-329bebb568cd None admin] RBAC: Adding query filter params (interface=public, service_id=3f90d574647a419eb919fa617715a706, region_id=RegionOne) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:06.877259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac2135-2bdb-468e-8a96-329bebb568cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:06.878613 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac2135-2bdb-468e-8a96-329bebb568cd None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:06.890821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 374/765] 10.222.0.59 () {56 vars in 1251 bytes} [Mon Aug 29 22:25:06 2022] GET /identity/v3/endpoints?service_id=3f90d574647a419eb919fa617715a706&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:06.890821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0d434-6ee1-44f0-b3b1-fca83170acab None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:06.890821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0d434-6ee1-44f0-b3b1-fca83170acab None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:06.890821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0d434-6ee1-44f0-b3b1-fca83170acab None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:06.890821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 375/766] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:06.890821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0d434-6ee1-44f0-b3b1-fca83170acab None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:06.892083 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:06.928665 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:06.933894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e75533-23aa-4270-9d8b-697957d185ec None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:06.976230 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33139795-9f29-44a6-80c3-82a004754ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:06.977975 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33139795-9f29-44a6-80c3-82a004754ce9 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:06.978404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33139795-9f29-44a6-80c3-82a004754ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:06.978838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33139795-9f29-44a6-80c3-82a004754ce9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:06.987347 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33139795-9f29-44a6-80c3-82a004754ce9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:06.989529 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33139795-9f29-44a6-80c3-82a004754ce9 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:07.005362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-33139795-9f29-44a6-80c3-82a004754ce9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:07.005362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:07.005362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:07.005362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:07.005362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:07.005362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:07.005362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:07.005362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:07.005362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:07.005362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:07.005362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:07.005362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:07.005362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:25:07.005362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:25:07.005362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:25:07.006443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:25:07.006443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:07.006443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:07.006443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:07.006443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:07.006443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:07.006443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:07.006443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:07.006443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:07.006443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:07.006443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:07.006443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:07.006443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:07.006443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:07.006443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:07.006443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:07.006443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:07.008546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:25:07.008546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:25:07.008546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:25:07.008546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:25:07.008546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:25:07.008546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:25:07.008546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:07.008546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:07.014565 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 392/767] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:07.014565 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33139795-9f29-44a6-80c3-82a004754ce9 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:07.018174 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0d434-6ee1-44f0-b3b1-fca83170acab None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:07.052760 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b2c5a-c2c7-40c6-aea3-72aa9410671b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:07.053366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b2c5a-c2c7-40c6-aea3-72aa9410671b None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:07.053642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b2c5a-c2c7-40c6-aea3-72aa9410671b None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:07.054532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b2c5a-c2c7-40c6-aea3-72aa9410671b None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:07.055201 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b2c5a-c2c7-40c6-aea3-72aa9410671b None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:07.055499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b2c5a-c2c7-40c6-aea3-72aa9410671b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:07.056887 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b2c5a-c2c7-40c6-aea3-72aa9410671b None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:07.073977 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 376/768] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:07.073977 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5133bb2-390d-482b-ad71-f0e091ec06a4 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:07.073977 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5133bb2-390d-482b-ad71-f0e091ec06a4 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:07.073977 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5133bb2-390d-482b-ad71-f0e091ec06a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:07.088717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-e5133bb2-390d-482b-ad71-f0e091ec06a4 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:07.089959 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-e5133bb2-390d-482b-ad71-f0e091ec06a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:07.131962 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33139795-9f29-44a6-80c3-82a004754ce9 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:07.162240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:07.165780 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 377/769] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:07.168770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb110c-2e13-4b81-b265-f4a7db6743bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:07.169379 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb110c-2e13-4b81-b265-f4a7db6743bf None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:07.169655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb110c-2e13-4b81-b265-f4a7db6743bf None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:07.169925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb110c-2e13-4b81-b265-f4a7db6743bf None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:07.170579 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb110c-2e13-4b81-b265-f4a7db6743bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:07.171930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb110c-2e13-4b81-b265-f4a7db6743bf None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:07.173220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5133bb2-390d-482b-ad71-f0e091ec06a4 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:07.188838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 393/770] 10.222.0.59 () {56 vars in 1087 bytes} [Mon Aug 29 22:25:07 2022] GET /identity/v3/services => generated 772 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:07.189789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb110c-2e13-4b81-b265-f4a7db6743bf None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:07.202727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:07.311118 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5133bb2-390d-482b-ad71-f0e091ec06a4 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:07.357434 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a1893-9dc9-4cfd-ac85-bad5a83e3329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:07.358539 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a1893-9dc9-4cfd-ac85-bad5a83e3329 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:07.359316 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a1893-9dc9-4cfd-ac85-bad5a83e3329 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:07.359791 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a1893-9dc9-4cfd-ac85-bad5a83e3329 None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:07.360732 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a1893-9dc9-4cfd-ac85-bad5a83e3329 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:07.361267 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a1893-9dc9-4cfd-ac85-bad5a83e3329 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:07.363123 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a1893-9dc9-4cfd-ac85-bad5a83e3329 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:07.380579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 378/771] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:25:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:07.387149 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a1893-9dc9-4cfd-ac85-bad5a83e3329 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:07.429856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb110c-2e13-4b81-b265-f4a7db6743bf None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:07.474604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7f249-969b-49fe-978f-a9dfb0e4ce57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:07.475238 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7f249-969b-49fe-978f-a9dfb0e4ce57 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:07.475520 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7f249-969b-49fe-978f-a9dfb0e4ce57 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:07.476353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7f249-969b-49fe-978f-a9dfb0e4ce57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:07.484944 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7f249-969b-49fe-978f-a9dfb0e4ce57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:07.486965 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7f249-969b-49fe-978f-a9dfb0e4ce57 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:07.501365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-44b7f249-969b-49fe-978f-a9dfb0e4ce57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:07.501365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:07.501365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:07.501365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:07.501365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:07.501365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:07.501365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:07.501365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:07.501365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:07.501365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:07.501365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:07.501365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:07.501365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:25:07.501365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:25:07.501365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:25:07.502325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:25:07.502325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:07.502325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:07.502325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:07.502325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:07.502325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:07.502325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:07.502325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:07.502325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:07.502325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:07.502325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:07.502325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:07.502325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:07.502325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:07.502325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:07.502325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:07.502325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:07.503276 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:25:07.503276 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:25:07.503276 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:25:07.503276 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:25:07.503276 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:25:07.503276 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:25:07.503276 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:07.503276 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:07.503276 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 394/772] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:07.503276 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7f249-969b-49fe-978f-a9dfb0e4ce57 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:07.528870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:07.549119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a1893-9dc9-4cfd-ac85-bad5a83e3329 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:07.581349 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:07.595915 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e2e52-30d0-4a1b-a173-a077a279f183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:07.596822 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e2e52-30d0-4a1b-a173-a077a279f183 None admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:07.597139 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e2e52-30d0-4a1b-a173-a077a279f183 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:07.597411 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e2e52-30d0-4a1b-a173-a077a279f183 None admin] PATH_INFO: `/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/7f21105679f245a19ea4850e6c010dbf/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:07.600274 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e2e52-30d0-4a1b-a173-a077a279f183 None admin] RBAC: Authorizing `identity:create_grant(project_id=7799820fc7c64fd6a99a0a9ccf1338e3, user_id=7f21105679f245a19ea4850e6c010dbf, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:07.602381 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e2e52-30d0-4a1b-a173-a077a279f183 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:07.635936 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7f249-969b-49fe-978f-a9dfb0e4ce57 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:07.674662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd3f91-e333-4d7c-8069-39213aef46c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:07.675563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd3f91-e333-4d7c-8069-39213aef46c7 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:07.675941 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd3f91-e333-4d7c-8069-39213aef46c7 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:07.676346 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd3f91-e333-4d7c-8069-39213aef46c7 None admin] PATH_INFO: `/v3/services/network` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:07.677131 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd3f91-e333-4d7c-8069-39213aef46c7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:07.678646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd3f91-e333-4d7c-8069-39213aef46c7 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:07.683786 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 379/773] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:25:07 2022] PUT /identity/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/7f21105679f245a19ea4850e6c010dbf/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:25:07.684954 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e2e52-30d0-4a1b-a173-a077a279f183 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:07.690081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-eedd3f91-e333-4d7c-8069-39213aef46c7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 22:25:07.690081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:07.690081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:07.690081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:07.690081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:07.690081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:07.690081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:07.690081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:07.690081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 22:25:07.690081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:07.690081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:07.690081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:07.690081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:07.690081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:07.690081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:07.690081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:07.690081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:07.691472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:07.691472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:07.691472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:07.691472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:07.691472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:07.691472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:07.691472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:07.691472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:07.691472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:07.691472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:07.691472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:07.691472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:07.691472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:07.691472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:07.691472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:07.691472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:07.692564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:07.692564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:07.692564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:07.692564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:07.692564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:07.692564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:07.692564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:07.692564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:07.692564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:07.692564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:07.692564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:07.692564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:07.692564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 22:25:07.692564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:07.694745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 395/774] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:07 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:07.705463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd3f91-e333-4d7c-8069-39213aef46c7 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:07.833163 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e2e52-30d0-4a1b-a173-a077a279f183 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:07.881599 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd3f91-e333-4d7c-8069-39213aef46c7 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:07.886035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b575f2-83a6-487a-8134-2d1b274fb35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:07.886655 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b575f2-83a6-487a-8134-2d1b274fb35f None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:07.886926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b575f2-83a6-487a-8134-2d1b274fb35f None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:07.887196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b575f2-83a6-487a-8134-2d1b274fb35f None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:07.887894 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b575f2-83a6-487a-8134-2d1b274fb35f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:07.889200 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b575f2-83a6-487a-8134-2d1b274fb35f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:07.889708 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b575f2-83a6-487a-8134-2d1b274fb35f None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:07.901086 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:07.905222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 380/775] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:25:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:07.907962 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5c954-4555-4cca-9fd2-8817ca85fd0f None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:07.908278 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5c954-4555-4cca-9fd2-8817ca85fd0f None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:07.908545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5c954-4555-4cca-9fd2-8817ca85fd0f None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:07.909960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 381/776] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:07.914600 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:07.914763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5c954-4555-4cca-9fd2-8817ca85fd0f None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:07.935151 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2123f-a1a8-4e53-a3fb-8ad54e4c0e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:07.935873 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2123f-a1a8-4e53-a3fb-8ad54e4c0e6b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:07.936113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2123f-a1a8-4e53-a3fb-8ad54e4c0e6b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:07.936420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2123f-a1a8-4e53-a3fb-8ad54e4c0e6b None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:07.937050 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2123f-a1a8-4e53-a3fb-8ad54e4c0e6b None admin] RBAC: Adding query filter params (name=network) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:07.937340 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2123f-a1a8-4e53-a3fb-8ad54e4c0e6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:07.938692 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2123f-a1a8-4e53-a3fb-8ad54e4c0e6b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:07.953609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 396/777] 10.222.0.59 () {56 vars in 1112 bytes} [Mon Aug 29 22:25:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:07.953609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc904d-87dc-4847-9501-5c4b30a1915b None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:07.953852 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc904d-87dc-4847-9501-5c4b30a1915b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:07.954433 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc904d-87dc-4847-9501-5c4b30a1915b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:07.973841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-dbcc904d-87dc-4847-9501-5c4b30a1915b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:07.975022 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-dbcc904d-87dc-4847-9501-5c4b30a1915b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:08.049004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 397/778] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:08.055656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc904d-87dc-4847-9501-5c4b30a1915b None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:08.057828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5c954-4555-4cca-9fd2-8817ca85fd0f None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:08.116960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294daf5c-ffdf-4112-a74b-19ac5e9aa696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:08.119954 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294daf5c-ffdf-4112-a74b-19ac5e9aa696 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:08.123071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294daf5c-ffdf-4112-a74b-19ac5e9aa696 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:08.123547 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294daf5c-ffdf-4112-a74b-19ac5e9aa696 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:08.131464 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294daf5c-ffdf-4112-a74b-19ac5e9aa696 None admin] RBAC: Adding query filter params (role.id=f91580fb2e5840bcb4f4054732e2e796, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=980157efae8f43b59dc063fda04c0348) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:08.131464 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294daf5c-ffdf-4112-a74b-19ac5e9aa696 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:08.131464 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294daf5c-ffdf-4112-a74b-19ac5e9aa696 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:08.154982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 382/779] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:25:07 2022] GET /identity/v3/role_assignments?user.id=980157efae8f43b59dc063fda04c0348&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=f91580fb2e5840bcb4f4054732e2e796 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:08.318258 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc904d-87dc-4847-9501-5c4b30a1915b None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:08.346212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:08.392042 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a70ad0-4ccd-44e4-8619-38165378e89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:08.396465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294daf5c-ffdf-4112-a74b-19ac5e9aa696 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:08.401763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a70ad0-4ccd-44e4-8619-38165378e89b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:08.401763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a70ad0-4ccd-44e4-8619-38165378e89b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:08.401763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a70ad0-4ccd-44e4-8619-38165378e89b None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:08.401763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a70ad0-4ccd-44e4-8619-38165378e89b None admin] RBAC: Adding query filter params (type=network) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:08.401763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a70ad0-4ccd-44e4-8619-38165378e89b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:08.401763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a70ad0-4ccd-44e4-8619-38165378e89b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:08.407216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 398/780] 10.222.0.59 () {56 vars in 1112 bytes} [Mon Aug 29 22:25:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:08.417412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a70ad0-4ccd-44e4-8619-38165378e89b None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:08.530407 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a70ad0-4ccd-44e4-8619-38165378e89b None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:08.574937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:08.575892 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d868f0b-a621-4aac-87a2-a97651e1192a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:08.576745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d868f0b-a621-4aac-87a2-a97651e1192a None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:08.576745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d868f0b-a621-4aac-87a2-a97651e1192a None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:08.577027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d868f0b-a621-4aac-87a2-a97651e1192a None admin] PATH_INFO: `/v3/endpoints` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:08.577737 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d868f0b-a621-4aac-87a2-a97651e1192a None admin] RBAC: Adding query filter params (interface=public, service_id=8c3773a6ccf746f892fa02577deb3a1a, region_id=RegionOne) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:08.578074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d868f0b-a621-4aac-87a2-a97651e1192a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:08.583881 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d868f0b-a621-4aac-87a2-a97651e1192a None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:08.609014 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 399/781] 10.222.0.59 () {56 vars in 1251 bytes} [Mon Aug 29 22:25:08 2022] GET /identity/v3/endpoints?service_id=8c3773a6ccf746f892fa02577deb3a1a&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:08.621837 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d868f0b-a621-4aac-87a2-a97651e1192a None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:08.653785 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294daf5c-ffdf-4112-a74b-19ac5e9aa696 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:08.703075 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:08.737429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc7fd5-f6bd-4f79-94ea-865fdae4e077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:08.737429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc7fd5-f6bd-4f79-94ea-865fdae4e077 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:08.737429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc7fd5-f6bd-4f79-94ea-865fdae4e077 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:08.737429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc7fd5-f6bd-4f79-94ea-865fdae4e077 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:08.762542 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc7fd5-f6bd-4f79-94ea-865fdae4e077 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:25:08.769653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-85bc7fd5-f6bd-4f79-94ea-865fdae4e077 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:25:08.769653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:08.769653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:08.769653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:08.769653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:08.769653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:08.769653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:08.769653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:08.769653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:08.769653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:08.769653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:08.769653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:08.769653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:25:08.769653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:25:08.769653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:25:08.769653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:25:08.770771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:25:08.770771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:25:08.770771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:25:08.770771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:25:08.770771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:08.770771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:08.770771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:08.770771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:08.770771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:08.770771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:08.770771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:08.770771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:08.770771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:08.770771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:08.770771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:08.770771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:08.771794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:08.771794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:08.771794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:08.771794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:08.771794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:25:08.771794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:25:08.771794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:25:08.771794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:25:08.771794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:25:08.771794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:25:08.771794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:25:08.771794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:08.780678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 383/782] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:25:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:08.785244 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc7fd5-f6bd-4f79-94ea-865fdae4e077 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:08.833179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d868f0b-a621-4aac-87a2-a97651e1192a None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:08.928178 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a1661-f451-44f4-ab4a-5d93112fafa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:08.929894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a1661-f451-44f4-ab4a-5d93112fafa6 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:08.930183 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a1661-f451-44f4-ab4a-5d93112fafa6 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:08.930488 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a1661-f451-44f4-ab4a-5d93112fafa6 None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:08.931326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a1661-f451-44f4-ab4a-5d93112fafa6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:08.936256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a1661-f451-44f4-ab4a-5d93112fafa6 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:08.952172 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc7fd5-f6bd-4f79-94ea-865fdae4e077 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:08.952368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 400/783] 10.222.0.59 () {56 vars in 1087 bytes} [Mon Aug 29 22:25:08 2022] GET /identity/v3/services => generated 772 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:08.998097 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:09.019818 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7484e-169d-410f-b55f-748575181491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:09.020460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7484e-169d-410f-b55f-748575181491 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:09.020745 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7484e-169d-410f-b55f-748575181491 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:09.021050 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7484e-169d-410f-b55f-748575181491 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:09.021717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7484e-169d-410f-b55f-748575181491 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:09.022012 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7484e-169d-410f-b55f-748575181491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:09.023882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7484e-169d-410f-b55f-748575181491 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:09.046982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 384/784] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:25:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:09.052377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a1661-f451-44f4-ab4a-5d93112fafa6 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:09.209300 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a1661-f451-44f4-ab4a-5d93112fafa6 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:09.213254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:09.248600 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67068c97-34e4-4fdd-af66-a590161c3fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:09.249228 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67068c97-34e4-4fdd-af66-a590161c3fb7 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:09.249497 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67068c97-34e4-4fdd-af66-a590161c3fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:09.249760 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67068c97-34e4-4fdd-af66-a590161c3fb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:09.263044 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67068c97-34e4-4fdd-af66-a590161c3fb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:09.264600 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67068c97-34e4-4fdd-af66-a590161c3fb7 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:09.279548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-67068c97-34e4-4fdd-af66-a590161c3fb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:09.279548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:09.279548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:09.279548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:09.279548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:09.279548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:09.279548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:09.279548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:09.279548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:09.279548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:09.279548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:09.279548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:09.279548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:09.279548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:09.279548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:09.279548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:09.280604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:09.280604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:09.280604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:09.280604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:09.280604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:09.280604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:09.280604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:09.280604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:09.280604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:09.280604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:09.280604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:09.280604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:09.280604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:09.280604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:09.280604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:09.280604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:09.280604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:09.281484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:09.281484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:09.281484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:09.281484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 401/785] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:09.286717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7484e-169d-410f-b55f-748575181491 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:09.405579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7484e-169d-410f-b55f-748575181491 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:09.440079 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70137b-2988-4bfd-a0bd-f3aff3463d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:09.440723 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70137b-2988-4bfd-a0bd-f3aff3463d19 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:09.441174 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70137b-2988-4bfd-a0bd-f3aff3463d19 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:09.441440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70137b-2988-4bfd-a0bd-f3aff3463d19 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:09.442129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70137b-2988-4bfd-a0bd-f3aff3463d19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:09.442427 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70137b-2988-4bfd-a0bd-f3aff3463d19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:09.443774 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70137b-2988-4bfd-a0bd-f3aff3463d19 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:09.457154 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 385/786] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:09.463323 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67068c97-34e4-4fdd-af66-a590161c3fb7 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:09.582281 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67068c97-34e4-4fdd-af66-a590161c3fb7 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:09.618777 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff7228-c3c0-483e-bad2-5ad79185809b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:09.619406 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff7228-c3c0-483e-bad2-5ad79185809b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:09.619678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff7228-c3c0-483e-bad2-5ad79185809b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:09.619945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff7228-c3c0-483e-bad2-5ad79185809b None admin] PATH_INFO: `/v3/users/swift` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:09.627992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff7228-c3c0-483e-bad2-5ad79185809b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:09.629499 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff7228-c3c0-483e-bad2-5ad79185809b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:09.641584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-7eff7228-c3c0-483e-bad2-5ad79185809b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:25:09.641584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:09.641584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:09.641584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:09.641584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:09.641584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:09.641584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:09.641584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:09.641584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:09.641584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:09.641584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:09.641584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:09.641584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:25:09.641584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:25:09.641584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:25:09.641584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:25:09.643934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:09.643934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:09.643934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:25:09.643934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:09.643934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:25:09.643934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:09.643934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:09.643934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:09.643934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:09.643934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:09.643934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:09.643934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:09.643934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:09.643934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:09.643934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:09.643934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:09.643934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:09.643934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:09.646327 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:09.646327 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:09.646327 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:25:09.646327 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:25:09.646327 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:25:09.646327 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:25:09.646327 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:25:09.646327 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:25:09.646327 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:25:09.646327 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:09.646327 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 402/787] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:25:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:09.652443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70137b-2988-4bfd-a0bd-f3aff3463d19 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:09.701310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:09.711652 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e70fb-a154-4aed-a667-5a0e6ceaa360 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:09.711990 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e70fb-a154-4aed-a667-5a0e6ceaa360 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:09.712211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e70fb-a154-4aed-a667-5a0e6ceaa360 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:09.713689 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 403/788] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:09.719511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c4a07-c73b-4386-8e23-b181972652b2 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:09.719774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c4a07-c73b-4386-8e23-b181972652b2 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:09.721809 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c4a07-c73b-4386-8e23-b181972652b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:09.743553 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-802c4a07-c73b-4386-8e23-b181972652b2 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:09.745590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-802c4a07-c73b-4386-8e23-b181972652b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:09.766215 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:09.784440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70137b-2988-4bfd-a0bd-f3aff3463d19 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:09.839245 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2e06e-12ef-401b-87da-c5b0ff525443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:09.840294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2e06e-12ef-401b-87da-c5b0ff525443 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:09.840790 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2e06e-12ef-401b-87da-c5b0ff525443 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:09.841264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2e06e-12ef-401b-87da-c5b0ff525443 None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:09.842367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2e06e-12ef-401b-87da-c5b0ff525443 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:09.842975 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2e06e-12ef-401b-87da-c5b0ff525443 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:09.844738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2e06e-12ef-401b-87da-c5b0ff525443 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:09.853194 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 404/789] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:09.879476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 386/790] 10.222.0.59 () {56 vars in 1135 bytes} [Mon Aug 29 22:25:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:09.886129 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c4a07-c73b-4386-8e23-b181972652b2 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:10.010345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c4a07-c73b-4386-8e23-b181972652b2 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:10.045860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3370652d-84a1-42fa-80e2-645bbb02def6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:10.046488 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370652d-84a1-42fa-80e2-645bbb02def6 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:10.046762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370652d-84a1-42fa-80e2-645bbb02def6 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:10.047046 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370652d-84a1-42fa-80e2-645bbb02def6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:10.058147 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3370652d-84a1-42fa-80e2-645bbb02def6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:10.059601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3370652d-84a1-42fa-80e2-645bbb02def6 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:10.066649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:10.072333 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d7091-d24d-4c02-89a3-893753fa8d38 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:10.072899 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d7091-d24d-4c02-89a3-893753fa8d38 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:10.073405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d7091-d24d-4c02-89a3-893753fa8d38 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:10.075696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-3370652d-84a1-42fa-80e2-645bbb02def6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:10.075696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:10.075696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:10.075696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:10.075696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:10.075696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:10.075696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:10.075696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:10.075696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:10.075696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:10.075696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:10.075696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:10.075696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:10.075696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:10.075696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:10.075696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:10.076636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:10.076636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:10.076636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:10.076636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:10.076636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:10.076636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:10.076636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:10.076636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:10.076636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:10.076636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:10.076636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:10.076636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:10.076636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:10.076636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:10.076636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:10.076636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:10.076636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:10.077547 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:10.077547 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:10.077547 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:10.079085 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 387/791] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:10.080487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 405/792] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:10.082328 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fabdb-d88f-47b5-840e-c80280e183e5 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:10.082610 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fabdb-d88f-47b5-840e-c80280e183e5 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:10.082924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fabdb-d88f-47b5-840e-c80280e183e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:10.086929 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3370652d-84a1-42fa-80e2-645bbb02def6 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:10.126215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-6d9fabdb-d88f-47b5-840e-c80280e183e5 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:10.127445 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6d9fabdb-d88f-47b5-840e-c80280e183e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:10.252077 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 388/793] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:10.263535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fabdb-d88f-47b5-840e-c80280e183e5 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:10.288905 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3370652d-84a1-42fa-80e2-645bbb02def6 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:10.331782 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97704411-9e47-40d5-a135-bd87caf785ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:10.332479 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97704411-9e47-40d5-a135-bd87caf785ea None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:10.332761 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97704411-9e47-40d5-a135-bd87caf785ea None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:10.333035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97704411-9e47-40d5-a135-bd87caf785ea None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:10.334194 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97704411-9e47-40d5-a135-bd87caf785ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:10.334502 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97704411-9e47-40d5-a135-bd87caf785ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:10.336913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97704411-9e47-40d5-a135-bd87caf785ea None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:10.342113 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:10.356858 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 406/794] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:10.362782 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97704411-9e47-40d5-a135-bd87caf785ea None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:10.476780 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fabdb-d88f-47b5-840e-c80280e183e5 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:10.492223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:10.548839 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c447be5-978c-4b27-81ea-6ff8739ea634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:10.549392 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c447be5-978c-4b27-81ea-6ff8739ea634 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:10.549670 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c447be5-978c-4b27-81ea-6ff8739ea634 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:10.549941 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c447be5-978c-4b27-81ea-6ff8739ea634 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:10.550532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c447be5-978c-4b27-81ea-6ff8739ea634 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:10.551908 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c447be5-978c-4b27-81ea-6ff8739ea634 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:10.571416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-2c447be5-978c-4b27-81ea-6ff8739ea634 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 22:25:10.571416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:10.571416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:10.571416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:10.571416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:10.571416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:10.571416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:10.571416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:10.571416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 22:25:10.571416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:10.571416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:10.571416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:10.571416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:10.571416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:10.571416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:10.571416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:10.571416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:10.572599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:10.572599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:10.572599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:10.572599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:10.572599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:10.572599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:10.572599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:10.572599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:10.572599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:10.572599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:10.572599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:10.572599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:10.572599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:10.572599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:10.572599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:10.572599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:10.573435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:10.573435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:10.573435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:10.573435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:10.573435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:10.573435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:10.573435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:10.573435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:10.573435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:10.573435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:10.573435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:10.573435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:10.573435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 22:25:10.573435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:10.574746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 389/795] 10.222.0.59 () {56 vars in 1129 bytes} [Mon Aug 29 22:25:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:10.579486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97704411-9e47-40d5-a135-bd87caf785ea None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:10.582095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c447be5-978c-4b27-81ea-6ff8739ea634 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:10.636931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad9152-3da8-4513-b9a8-ecf6d3ee5415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:10.637658 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad9152-3da8-4513-b9a8-ecf6d3ee5415 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:10.637941 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad9152-3da8-4513-b9a8-ecf6d3ee5415 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:10.638231 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad9152-3da8-4513-b9a8-ecf6d3ee5415 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:10.646969 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad9152-3da8-4513-b9a8-ecf6d3ee5415 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:10.649563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad9152-3da8-4513-b9a8-ecf6d3ee5415 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:10.664993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-27ad9152-3da8-4513-b9a8-ecf6d3ee5415 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:10.664993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:10.664993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:10.664993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:10.664993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:10.664993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:10.664993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:10.664993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:10.664993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:10.664993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:10.664993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:10.664993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:10.664993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:25:10.664993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:25:10.664993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:25:10.666176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:25:10.666176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:10.666176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:10.666176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:10.666176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:10.666176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:10.666176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:10.666176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:10.666176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:10.666176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:10.666176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:10.666176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:10.666176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:10.666176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:10.666176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:10.666176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:10.666176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:10.667212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:25:10.667212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:25:10.667212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:25:10.667212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:25:10.667212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:25:10.667212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:25:10.667212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:10.667212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:10.667212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 407/796] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:10.667212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad9152-3da8-4513-b9a8-ecf6d3ee5415 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:10.718290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:10.723394 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c447be5-978c-4b27-81ea-6ff8739ea634 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:10.769307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccb7a0-87db-4c91-9e3e-7ae5f3206c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:10.769911 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccb7a0-87db-4c91-9e3e-7ae5f3206c69 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:10.770210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccb7a0-87db-4c91-9e3e-7ae5f3206c69 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:10.770463 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccb7a0-87db-4c91-9e3e-7ae5f3206c69 None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:10.771114 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccb7a0-87db-4c91-9e3e-7ae5f3206c69 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:10.771417 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccb7a0-87db-4c91-9e3e-7ae5f3206c69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:10.773864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccb7a0-87db-4c91-9e3e-7ae5f3206c69 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:10.784192 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 390/797] 10.222.0.59 () {56 vars in 1124 bytes} [Mon Aug 29 22:25:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:10.784192 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccb7a0-87db-4c91-9e3e-7ae5f3206c69 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:10.801028 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad9152-3da8-4513-b9a8-ecf6d3ee5415 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:10.827995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:10.850903 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ff3d1-c74f-478a-b54c-33682b135ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:10.852209 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ff3d1-c74f-478a-b54c-33682b135ad6 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:10.852804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ff3d1-c74f-478a-b54c-33682b135ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:10.853305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ff3d1-c74f-478a-b54c-33682b135ad6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:10.865992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ff3d1-c74f-478a-b54c-33682b135ad6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:25:10.871751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-208ff3d1-c74f-478a-b54c-33682b135ad6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:25:10.871751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:10.871751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:10.871751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:10.871751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:10.871751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:10.871751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:10.871751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:10.871751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:10.871751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:10.871751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:10.871751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:10.871751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:25:10.871751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:25:10.871751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:25:10.871751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:25:10.872912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:25:10.872912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:25:10.872912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:25:10.872912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:25:10.872912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:10.872912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:10.872912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:10.872912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:10.872912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:10.872912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:10.872912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:10.872912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:10.872912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:10.872912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:10.872912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:10.872912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:10.873796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:10.873796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:10.873796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:10.873796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:10.873796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:25:10.873796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:25:10.873796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:25:10.873796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:25:10.873796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:25:10.873796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:25:10.873796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:25:10.873796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:10.877944 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 408/798] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:25:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:10.878632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ff3d1-c74f-478a-b54c-33682b135ad6 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:10.904035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccb7a0-87db-4c91-9e3e-7ae5f3206c69 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:10.947101 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ca10b-25e2-4c8d-b41e-b44b78d4b1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:10.947722 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ca10b-25e2-4c8d-b41e-b44b78d4b1bc None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:10.948026 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ca10b-25e2-4c8d-b41e-b44b78d4b1bc None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:10.948322 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ca10b-25e2-4c8d-b41e-b44b78d4b1bc None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:10.949014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ca10b-25e2-4c8d-b41e-b44b78d4b1bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:10.949311 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ca10b-25e2-4c8d-b41e-b44b78d4b1bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:10.950723 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ca10b-25e2-4c8d-b41e-b44b78d4b1bc None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:10.970376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 391/799] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:25:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:10.970376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abce51-df5f-437c-8998-96f8a397a115 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:10.970376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abce51-df5f-437c-8998-96f8a397a115 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:10.970376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abce51-df5f-437c-8998-96f8a397a115 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:10.970376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 392/800] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:10.972012 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abce51-df5f-437c-8998-96f8a397a115 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:11.005400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:11.011403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ff3d1-c74f-478a-b54c-33682b135ad6 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:11.049721 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42a3ea-30ad-455a-85d2-ab509d8a6d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:11.050414 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42a3ea-30ad-455a-85d2-ab509d8a6d22 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:11.050691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42a3ea-30ad-455a-85d2-ab509d8a6d22 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:11.050956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42a3ea-30ad-455a-85d2-ab509d8a6d22 None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:11.051630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42a3ea-30ad-455a-85d2-ab509d8a6d22 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:11.051873 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42a3ea-30ad-455a-85d2-ab509d8a6d22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:11.053254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42a3ea-30ad-455a-85d2-ab509d8a6d22 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:11.058843 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 409/801] 10.222.0.59 () {56 vars in 1124 bytes} [Mon Aug 29 22:25:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:11.063718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f41a5e-54cc-422b-a370-0772e9e2869b None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:11.063718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f41a5e-54cc-422b-a370-0772e9e2869b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:11.063718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f41a5e-54cc-422b-a370-0772e9e2869b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:11.080019 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-b5f41a5e-54cc-422b-a370-0772e9e2869b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:11.081230 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b5f41a5e-54cc-422b-a370-0772e9e2869b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:11.116739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abce51-df5f-437c-8998-96f8a397a115 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:11.125282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:11.159098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23252b4-41e1-4db4-94c8-34488b9e7116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:11.159724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23252b4-41e1-4db4-94c8-34488b9e7116 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:11.159999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23252b4-41e1-4db4-94c8-34488b9e7116 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:11.160317 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23252b4-41e1-4db4-94c8-34488b9e7116 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:11.160967 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23252b4-41e1-4db4-94c8-34488b9e7116 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:11.161265 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23252b4-41e1-4db4-94c8-34488b9e7116 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:11.162641 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23252b4-41e1-4db4-94c8-34488b9e7116 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:11.172461 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 410/802] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:11.172461 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f41a5e-54cc-422b-a370-0772e9e2869b None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:11.172708 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 393/803] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:25:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:11.173378 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23252b4-41e1-4db4-94c8-34488b9e7116 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:11.283518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:11.307258 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23252b4-41e1-4db4-94c8-34488b9e7116 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:11.309274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f41a5e-54cc-422b-a370-0772e9e2869b None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:11.347175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301765cd-a89c-47f4-a5f1-fe9a11ff8ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:11.347849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301765cd-a89c-47f4-a5f1-fe9a11ff8ff3 None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:11.348065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301765cd-a89c-47f4-a5f1-fe9a11ff8ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:11.348380 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301765cd-a89c-47f4-a5f1-fe9a11ff8ff3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:11.349058 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301765cd-a89c-47f4-a5f1-fe9a11ff8ff3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:11.350569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301765cd-a89c-47f4-a5f1-fe9a11ff8ff3 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:11.357531 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f773f33-0aad-466b-9db4-a213a408469b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:11.358642 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f773f33-0aad-466b-9db4-a213a408469b None admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:11.359075 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f773f33-0aad-466b-9db4-a213a408469b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:11.359342 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f773f33-0aad-466b-9db4-a213a408469b None admin] PATH_INFO: `/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/f1f0503ae1b24ce6a8adb3cfd7973f14/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:11.361284 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f773f33-0aad-466b-9db4-a213a408469b None admin] RBAC: Authorizing `identity:create_grant(project_id=7799820fc7c64fd6a99a0a9ccf1338e3, user_id=f1f0503ae1b24ce6a8adb3cfd7973f14, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:11.362708 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f773f33-0aad-466b-9db4-a213a408469b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:11.474236 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 394/804] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:25:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:25:11.475142 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301765cd-a89c-47f4-a5f1-fe9a11ff8ff3 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:11.502914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 411/805] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:25:11 2022] PUT /identity/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/f1f0503ae1b24ce6a8adb3cfd7973f14/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:25:11.504374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4723a-9ad3-4e78-8bc8-909083704134 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:11.504653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4723a-9ad3-4e78-8bc8-909083704134 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:11.509930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4723a-9ad3-4e78-8bc8-909083704134 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:11.509930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 412/806] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:11.509930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4723a-9ad3-4e78-8bc8-909083704134 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:11.568476 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:11.658198 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301765cd-a89c-47f4-a5f1-fe9a11ff8ff3 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:11.662761 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4723a-9ad3-4e78-8bc8-909083704134 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:11.712688 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcdfaf1-0654-4d4a-a160-84bb498ee0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:11.713349 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdfaf1-0654-4d4a-a160-84bb498ee0fb None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:11.713625 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdfaf1-0654-4d4a-a160-84bb498ee0fb None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:11.713894 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdfaf1-0654-4d4a-a160-84bb498ee0fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:11.717903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:11.730980 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcdfaf1-0654-4d4a-a160-84bb498ee0fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:11.733631 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcdfaf1-0654-4d4a-a160-84bb498ee0fb None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:11.750779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-bbcdfaf1-0654-4d4a-a160-84bb498ee0fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:11.750779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:11.750779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:11.750779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:11.750779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:11.750779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:11.750779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:11.750779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:11.750779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:11.750779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:11.750779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:11.750779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:11.750779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:11.750779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:11.750779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:11.750779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:11.751803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:11.751803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:11.751803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:11.751803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:11.751803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:11.751803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:11.751803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:11.751803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:11.751803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:11.751803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:11.751803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:11.751803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:11.751803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:11.751803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:11.751803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:11.751803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:11.751803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:11.753461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:11.753461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:11.753461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:11.753461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 395/807] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:11.753841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76c40e-2549-40a9-8bac-c0fa94a678fd None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:11.760376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76c40e-2549-40a9-8bac-c0fa94a678fd None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:11.760376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76c40e-2549-40a9-8bac-c0fa94a678fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:11.780548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-cc76c40e-2549-40a9-8bac-c0fa94a678fd None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:11.782015 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-cc76c40e-2549-40a9-8bac-c0fa94a678fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:11.836636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e17c5-1005-452e-9cb1-95c338c662b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:11.837175 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e17c5-1005-452e-9cb1-95c338c662b0 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:11.837444 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e17c5-1005-452e-9cb1-95c338c662b0 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:11.837714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e17c5-1005-452e-9cb1-95c338c662b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:11.852172 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957e17c5-1005-452e-9cb1-95c338c662b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:25:11.858554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-957e17c5-1005-452e-9cb1-95c338c662b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:25:11.858554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:11.858554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:11.858554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:11.858554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:11.858554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:11.858554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:11.858554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:11.858554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:11.858554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:11.858554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:11.858554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:11.858554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:25:11.858554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:25:11.858554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:25:11.858554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:25:11.859700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:25:11.859700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:25:11.859700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:25:11.859700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:25:11.859700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:11.859700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:11.859700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:11.859700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:11.859700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:11.859700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:11.859700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:11.859700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:11.859700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:11.859700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:11.859700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:11.859700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:11.860745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:11.860745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:11.860745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:11.860745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:11.860745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:25:11.860745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:25:11.860745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:25:11.860745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:25:11.860745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:25:11.860745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:25:11.860745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:25:11.860745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:11.860745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 413/808] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:25:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:11.867690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e17c5-1005-452e-9cb1-95c338c662b0 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:11.881520 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 396/809] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:11.888419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76c40e-2549-40a9-8bac-c0fa94a678fd None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:12.028900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76c40e-2549-40a9-8bac-c0fa94a678fd None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:12.064182 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:12.066351 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:12.077932 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3570da-0d9f-4690-9aac-8ae6364b5079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:12.082222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3570da-0d9f-4690-9aac-8ae6364b5079 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:12.082222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3570da-0d9f-4690-9aac-8ae6364b5079 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:12.082222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3570da-0d9f-4690-9aac-8ae6364b5079 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:12.082222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3570da-0d9f-4690-9aac-8ae6364b5079 None admin] RBAC: Adding query filter params (name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:12.082222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3570da-0d9f-4690-9aac-8ae6364b5079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:12.082222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3570da-0d9f-4690-9aac-8ae6364b5079 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:12.086770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e17c5-1005-452e-9cb1-95c338c662b0 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:12.100385 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 397/810] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:25:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:12.106885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3570da-0d9f-4690-9aac-8ae6364b5079 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:12.165538 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113ee9d-6c4e-4acf-90db-d4bab357fb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:12.166153 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113ee9d-6c4e-4acf-90db-d4bab357fb92 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:12.166429 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113ee9d-6c4e-4acf-90db-d4bab357fb92 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:12.166699 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113ee9d-6c4e-4acf-90db-d4bab357fb92 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:12.167346 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113ee9d-6c4e-4acf-90db-d4bab357fb92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:12.167646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113ee9d-6c4e-4acf-90db-d4bab357fb92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:12.169185 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113ee9d-6c4e-4acf-90db-d4bab357fb92 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:12.193829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 414/811] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:12.199776 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113ee9d-6c4e-4acf-90db-d4bab357fb92 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:12.310123 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3570da-0d9f-4690-9aac-8ae6364b5079 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:12.347847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113ee9d-6c4e-4acf-90db-d4bab357fb92 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:12.350947 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1d2ff-de2f-4996-b1f6-aaf1128ac0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:12.352275 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1d2ff-de2f-4996-b1f6-aaf1128ac0c1 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:12.352894 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1d2ff-de2f-4996-b1f6-aaf1128ac0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:12.353611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1d2ff-de2f-4996-b1f6-aaf1128ac0c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:12.365925 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c1d2ff-de2f-4996-b1f6-aaf1128ac0c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:12.368954 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c1d2ff-de2f-4996-b1f6-aaf1128ac0c1 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:12.384189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-f7c1d2ff-de2f-4996-b1f6-aaf1128ac0c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:12.384189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:12.384189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:12.384189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:12.384189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:12.384189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:12.384189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:12.384189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:12.384189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:12.384189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:12.384189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:12.384189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:12.384189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:12.384189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:12.384189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:12.384189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:12.385256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:12.385256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:12.385256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:12.385256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:12.385256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:12.385256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:12.385256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:12.385256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:12.385256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:12.385256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:12.385256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:12.385256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:12.385256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:12.385256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:12.385256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:12.385256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:12.385256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:12.386147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:12.386147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:12.386147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:12.393388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 398/812] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:12.393540 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df32d2-de68-428e-b012-0accab2f79f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:12.393540 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df32d2-de68-428e-b012-0accab2f79f5 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:12.393540 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df32d2-de68-428e-b012-0accab2f79f5 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:12.393540 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df32d2-de68-428e-b012-0accab2f79f5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:12.395281 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1d2ff-de2f-4996-b1f6-aaf1128ac0c1 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:12.406790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df32d2-de68-428e-b012-0accab2f79f5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:12.408650 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df32d2-de68-428e-b012-0accab2f79f5 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:12.422196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-58df32d2-de68-428e-b012-0accab2f79f5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:25:12.422196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:12.422196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:12.422196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:12.422196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:12.422196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:12.422196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:12.422196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:12.422196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:12.422196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:12.422196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:12.422196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:12.422196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:25:12.422196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:25:12.422196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:25:12.422196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:25:12.423190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:12.423190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:12.423190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:25:12.423190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:12.423190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:25:12.423190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:12.423190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:12.423190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:12.423190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:12.423190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:12.423190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:12.423190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:12.423190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:12.423190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:12.423190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:12.423190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:12.423190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:12.423190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:12.424451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:12.424451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:12.424451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:25:12.424451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:25:12.424451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:25:12.424451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:25:12.424451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:25:12.424451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:25:12.424451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:25:12.424451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:12.432241 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 415/813] 10.222.0.59 () {56 vars in 1093 bytes} [Mon Aug 29 22:25:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:12.433509 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df32d2-de68-428e-b012-0accab2f79f5 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:12.450980 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:12.472625 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:12.533603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1d2ff-de2f-4996-b1f6-aaf1128ac0c1 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:12.565950 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df32d2-de68-428e-b012-0accab2f79f5 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:12.586449 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855276c8-95b0-4c57-843a-730c7f45c9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:12.588166 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855276c8-95b0-4c57-843a-730c7f45c9c7 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:12.588662 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855276c8-95b0-4c57-843a-730c7f45c9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:12.589240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855276c8-95b0-4c57-843a-730c7f45c9c7 None admin] PATH_INFO: `/v3/services/network` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:12.590025 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855276c8-95b0-4c57-843a-730c7f45c9c7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:12.591629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855276c8-95b0-4c57-843a-730c7f45c9c7 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:12.602763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-855276c8-95b0-4c57-843a-730c7f45c9c7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 22:25:12.602763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:12.602763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:12.602763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:12.602763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:12.602763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:12.602763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:12.602763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:12.602763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 22:25:12.602763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:12.602763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:12.602763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:12.602763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:12.602763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:12.602763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:12.602763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:12.602763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:12.603797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:12.603797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:12.603797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:12.603797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:12.603797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:12.603797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:12.603797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:12.603797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:12.603797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:12.603797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:12.603797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:12.603797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:12.603797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:12.603797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:12.603797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:12.603797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:12.604649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:12.604649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:12.604649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:12.604649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:12.604649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:12.604649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:12.604649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:12.604649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:12.604649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:12.604649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:12.604649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:12.604649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:12.604649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 22:25:12.604649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:12.605375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c109d9-9570-4d08-bcf8-0b9e4550460c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:12.606199 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c109d9-9570-4d08-bcf8-0b9e4550460c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:12.606546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c109d9-9570-4d08-bcf8-0b9e4550460c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:12.606881 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c109d9-9570-4d08-bcf8-0b9e4550460c None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:12.607699 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c109d9-9570-4d08-bcf8-0b9e4550460c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:12.608247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c109d9-9570-4d08-bcf8-0b9e4550460c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:12.608812 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 399/814] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:12 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:12.609804 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855276c8-95b0-4c57-843a-730c7f45c9c7 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:12.615262 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c109d9-9570-4d08-bcf8-0b9e4550460c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:12.628998 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 416/815] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:12.628998 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c109d9-9570-4d08-bcf8-0b9e4550460c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:12.704941 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:12.730882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:12.738171 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855276c8-95b0-4c57-843a-730c7f45c9c7 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:12.742821 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c109d9-9570-4d08-bcf8-0b9e4550460c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:12.783595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc881a1-7de0-4edb-81a5-3649b13c636b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:12.784344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc881a1-7de0-4edb-81a5-3649b13c636b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:12.784692 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc881a1-7de0-4edb-81a5-3649b13c636b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:12.785033 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc881a1-7de0-4edb-81a5-3649b13c636b None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:12.785828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc881a1-7de0-4edb-81a5-3649b13c636b None admin] RBAC: Adding query filter params (name=network) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:12.786132 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14790588-fcf3-4f78-8a0a-de2ff4cf2ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:12.786271 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc881a1-7de0-4edb-81a5-3649b13c636b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:12.787064 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14790588-fcf3-4f78-8a0a-de2ff4cf2ddf None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:12.787511 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14790588-fcf3-4f78-8a0a-de2ff4cf2ddf None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:12.787768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc881a1-7de0-4edb-81a5-3649b13c636b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:12.788323 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14790588-fcf3-4f78-8a0a-de2ff4cf2ddf None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:12.789385 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14790588-fcf3-4f78-8a0a-de2ff4cf2ddf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:12.789869 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14790588-fcf3-4f78-8a0a-de2ff4cf2ddf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:12.791613 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14790588-fcf3-4f78-8a0a-de2ff4cf2ddf None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:12.798182 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 417/816] 10.222.0.59 () {56 vars in 1112 bytes} [Mon Aug 29 22:25:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:12.799455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc881a1-7de0-4edb-81a5-3649b13c636b None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:12.818490 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 400/817] 10.222.0.59 () {56 vars in 1133 bytes} [Mon Aug 29 22:25:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:12.820045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14790588-fcf3-4f78-8a0a-de2ff4cf2ddf None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:12.914025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc881a1-7de0-4edb-81a5-3649b13c636b None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:12.943582 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14790588-fcf3-4f78-8a0a-de2ff4cf2ddf None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:12.953766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624d063-f456-4b60-b779-6b5f90e73c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:12.954527 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624d063-f456-4b60-b779-6b5f90e73c5b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:12.954906 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624d063-f456-4b60-b779-6b5f90e73c5b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:12.955274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624d063-f456-4b60-b779-6b5f90e73c5b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:12.958213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:12.964063 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624d063-f456-4b60-b779-6b5f90e73c5b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:12.965691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624d063-f456-4b60-b779-6b5f90e73c5b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:12.978770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-d624d063-f456-4b60-b779-6b5f90e73c5b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:25:12.978770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:12.978770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:12.978770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:12.978770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:12.978770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:12.978770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:12.978770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:12.978770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:12.978770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:12.978770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:12.978770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:12.978770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:25:12.978770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:25:12.978770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:25:12.978770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:25:12.979957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:12.979957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:12.979957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:25:12.979957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:12.979957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:25:12.979957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:12.979957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:12.979957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:12.979957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:12.979957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:12.979957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:12.979957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:12.979957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:12.979957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:12.979957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:12.979957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:12.979957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:12.979957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:12.980978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:12.980978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:12.980978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:25:12.980978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:25:12.980978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:25:12.980978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:25:12.980978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:25:12.980978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:25:12.980978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:25:12.980978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:12.986032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 418/818] 10.222.0.59 () {56 vars in 1117 bytes} [Mon Aug 29 22:25:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:12.987113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624d063-f456-4b60-b779-6b5f90e73c5b None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:12.989654 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b51933-e108-463a-b41a-86c3834c2b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:12.990498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b51933-e108-463a-b41a-86c3834c2b31 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:12.991031 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b51933-e108-463a-b41a-86c3834c2b31 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:12.991466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b51933-e108-463a-b41a-86c3834c2b31 None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:12.993266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b51933-e108-463a-b41a-86c3834c2b31 None admin] RBAC: Adding query filter params (type=network) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:12.994481 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b51933-e108-463a-b41a-86c3834c2b31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:12.996016 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b51933-e108-463a-b41a-86c3834c2b31 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:13.003371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 401/819] 10.222.0.59 () {56 vars in 1112 bytes} [Mon Aug 29 22:25:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:13.004579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b51933-e108-463a-b41a-86c3834c2b31 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:13.028473 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:13.103912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624d063-f456-4b60-b779-6b5f90e73c5b None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:13.120690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b51933-e108-463a-b41a-86c3834c2b31 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:13.141875 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05677505-1094-492e-bcee-a94b5d6bc12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:13.142578 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05677505-1094-492e-bcee-a94b5d6bc12b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:13.142924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05677505-1094-492e-bcee-a94b5d6bc12b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:13.143266 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05677505-1094-492e-bcee-a94b5d6bc12b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:13.160852 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbd116-38fb-4753-97c6-e3f2dd4c8e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:13.161911 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbd116-38fb-4753-97c6-e3f2dd4c8e8e None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:13.162404 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbd116-38fb-4753-97c6-e3f2dd4c8e8e None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:13.162923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbd116-38fb-4753-97c6-e3f2dd4c8e8e None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:13.164248 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fbd116-38fb-4753-97c6-e3f2dd4c8e8e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:13.164840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fbd116-38fb-4753-97c6-e3f2dd4c8e8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:13.166457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fbd116-38fb-4753-97c6-e3f2dd4c8e8e None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:13.187534 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 402/820] 10.222.0.59 () {56 vars in 1149 bytes} [Mon Aug 29 22:25:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:13.188916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbd116-38fb-4753-97c6-e3f2dd4c8e8e None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:13.216776 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:13.223540 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05677505-1094-492e-bcee-a94b5d6bc12b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:13.225024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05677505-1094-492e-bcee-a94b5d6bc12b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:13.280620 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:13.309070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbd116-38fb-4753-97c6-e3f2dd4c8e8e None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:13.343819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 419/821] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:25:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:25:13.343819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05677505-1094-492e-bcee-a94b5d6bc12b None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:13.367968 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aedc0b0-dac8-4ef7-9e55-21e1be0bcf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:13.368685 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedc0b0-dac8-4ef7-9e55-21e1be0bcf5b None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:13.368914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedc0b0-dac8-4ef7-9e55-21e1be0bcf5b None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:13.369244 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedc0b0-dac8-4ef7-9e55-21e1be0bcf5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:13.370084 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aedc0b0-dac8-4ef7-9e55-21e1be0bcf5b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:13.371624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aedc0b0-dac8-4ef7-9e55-21e1be0bcf5b None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:13.458549 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:13.465762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05677505-1094-492e-bcee-a94b5d6bc12b None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:13.497126 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 403/822] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:25:13 2022] POST /identity/v3/endpoints => generated 337 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:25:13.498696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aedc0b0-dac8-4ef7-9e55-21e1be0bcf5b None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:13.543478 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537e82f-8c5f-4312-a1f0-f9cea2f21205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:13.545419 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537e82f-8c5f-4312-a1f0-f9cea2f21205 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:13.545702 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537e82f-8c5f-4312-a1f0-f9cea2f21205 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:13.547377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537e82f-8c5f-4312-a1f0-f9cea2f21205 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:13.576555 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537e82f-8c5f-4312-a1f0-f9cea2f21205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:13.578046 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537e82f-8c5f-4312-a1f0-f9cea2f21205 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:13.604772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-1537e82f-8c5f-4312-a1f0-f9cea2f21205 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:13.604772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:13.604772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:13.604772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:13.604772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:13.604772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:13.604772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:13.604772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:13.604772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:13.604772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:13.604772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:13.604772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:13.604772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:13.604772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:13.604772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:13.604772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:13.605817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:13.605817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:13.605817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:13.605817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:13.605817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:13.605817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:13.605817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:13.605817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:13.605817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:13.605817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:13.605817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:13.605817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:13.605817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:13.605817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:13.605817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:13.605817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:13.605817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:13.606703 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:13.606703 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:13.606703 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:13.609100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 420/823] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:13.615242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6b51c-49d4-4d5e-afcf-0b71eb925ff6 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:13.616644 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6b51c-49d4-4d5e-afcf-0b71eb925ff6 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:13.617294 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6b51c-49d4-4d5e-afcf-0b71eb925ff6 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:13.619543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 421/824] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:13 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:13.620641 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6b51c-49d4-4d5e-afcf-0b71eb925ff6 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:13.677831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aedc0b0-dac8-4ef7-9e55-21e1be0bcf5b None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:13.719966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:13.731099 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b7245-2b0d-4016-8302-8e512c0f4bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:13.731749 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b7245-2b0d-4016-8302-8e512c0f4bc9 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:13.732022 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b7245-2b0d-4016-8302-8e512c0f4bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:13.732360 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b7245-2b0d-4016-8302-8e512c0f4bc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:13.745025 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b7245-2b0d-4016-8302-8e512c0f4bc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:13.746520 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b7245-2b0d-4016-8302-8e512c0f4bc9 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:13.762507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-b00b7245-2b0d-4016-8302-8e512c0f4bc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:13.762507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:13.762507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:13.762507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:13.762507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:13.762507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:13.762507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:13.762507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:13.762507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:13.762507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:13.762507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:13.762507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:13.762507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:13.762507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:13.762507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:13.762507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:13.763545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:13.763545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:13.763545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:13.763545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:13.763545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:13.763545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:13.763545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:13.763545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:13.763545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:13.763545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:13.763545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:13.763545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:13.763545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:13.763545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:13.763545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:13.763545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:13.763545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:13.764675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:13.764675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:13.764675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:13.764845 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6b51c-49d4-4d5e-afcf-0b71eb925ff6 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:13.765238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 404/825] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:13.765238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6a8e5-5241-4bd1-99a2-1808e9a0afc5 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:13.765238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6a8e5-5241-4bd1-99a2-1808e9a0afc5 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:13.765427 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6a8e5-5241-4bd1-99a2-1808e9a0afc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:13.783059 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-0da6a8e5-5241-4bd1-99a2-1808e9a0afc5 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:13.784314 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-0da6a8e5-5241-4bd1-99a2-1808e9a0afc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:13.813111 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444eb95-80d5-48f9-a12a-ccf21dd77c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:13.813700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444eb95-80d5-48f9-a12a-ccf21dd77c4d None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:13.813968 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444eb95-80d5-48f9-a12a-ccf21dd77c4d None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:13.814227 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444eb95-80d5-48f9-a12a-ccf21dd77c4d None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:13.814853 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444eb95-80d5-48f9-a12a-ccf21dd77c4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:13.815223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444eb95-80d5-48f9-a12a-ccf21dd77c4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:13.816620 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444eb95-80d5-48f9-a12a-ccf21dd77c4d None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:13.828667 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 422/826] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:13.829853 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d41fd-8a40-48d9-a289-50f26986c7c0 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:13.830125 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d41fd-8a40-48d9-a289-50f26986c7c0 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:13.830386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d41fd-8a40-48d9-a289-50f26986c7c0 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:13.831554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 423/827] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:13.832304 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d41fd-8a40-48d9-a289-50f26986c7c0 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:13.836664 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:13.858066 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 405/828] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:13.860800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6a8e5-5241-4bd1-99a2-1808e9a0afc5 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:13.951591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d41fd-8a40-48d9-a289-50f26986c7c0 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:13.980675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6a8e5-5241-4bd1-99a2-1808e9a0afc5 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:13.994903 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab669f7-1879-4bff-a8dc-d40af41eadae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:13.995521 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab669f7-1879-4bff-a8dc-d40af41eadae None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:13.995861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab669f7-1879-4bff-a8dc-d40af41eadae None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:13.996164 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab669f7-1879-4bff-a8dc-d40af41eadae None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:13.996789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab669f7-1879-4bff-a8dc-d40af41eadae None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:13.997184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab669f7-1879-4bff-a8dc-d40af41eadae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:13.998434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab669f7-1879-4bff-a8dc-d40af41eadae None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:13.999062 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:14.012610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 424/829] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:14.013698 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b1218-5095-4468-977b-d8851efda7cc None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:14.013943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b1218-5095-4468-977b-d8851efda7cc None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:14.014184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b1218-5095-4468-977b-d8851efda7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:14.027455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c51fe0-0340-4333-9839-7f9a267665ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:14.028088 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c51fe0-0340-4333-9839-7f9a267665ab None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:14.028408 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c51fe0-0340-4333-9839-7f9a267665ab None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:14.028675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c51fe0-0340-4333-9839-7f9a267665ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:14.034465 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-4a9b1218-5095-4468-977b-d8851efda7cc None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:14.035642 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-4a9b1218-5095-4468-977b-d8851efda7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:14.035773 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c51fe0-0340-4333-9839-7f9a267665ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:14.037170 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c51fe0-0340-4333-9839-7f9a267665ab None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:14.102970 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:14.120169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 425/830] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:14.120169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b1218-5095-4468-977b-d8851efda7cc None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:14.164867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-d8c51fe0-0340-4333-9839-7f9a267665ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:14.164867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:14.164867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:14.164867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:14.164867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:14.164867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:14.164867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:14.164867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:14.164867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:14.164867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:14.164867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:14.164867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:14.164867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:25:14.164867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:25:14.164867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:25:14.165842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:25:14.165842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:14.165842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:14.165842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:14.165842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:14.165842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:14.165842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:14.165842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:14.165842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:14.165842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:14.165842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:14.165842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:14.165842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:14.165842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:14.165842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:14.165842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:14.165842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:14.166714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:25:14.166714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:25:14.166714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:25:14.166714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:25:14.166714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:25:14.166714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:25:14.166714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:14.166714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:14.166714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 406/831] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:14.167243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c51fe0-0340-4333-9839-7f9a267665ab None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:14.245260 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b1218-5095-4468-977b-d8851efda7cc None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:14.277897 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c51fe0-0340-4333-9839-7f9a267665ab None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:14.291960 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc157be-d38b-40ad-afb4-c5ad5690ac19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:14.292811 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc157be-d38b-40ad-afb4-c5ad5690ac19 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:14.293266 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc157be-d38b-40ad-afb4-c5ad5690ac19 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:14.293680 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc157be-d38b-40ad-afb4-c5ad5690ac19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:14.300785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc157be-d38b-40ad-afb4-c5ad5690ac19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:14.302398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc157be-d38b-40ad-afb4-c5ad5690ac19 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:14.314069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b07c1d-b6df-4407-9171-28822b8a0c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:14.314696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b07c1d-b6df-4407-9171-28822b8a0c82 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:14.314971 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b07c1d-b6df-4407-9171-28822b8a0c82 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:14.315237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b07c1d-b6df-4407-9171-28822b8a0c82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:14.316655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-6fc157be-d38b-40ad-afb4-c5ad5690ac19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:14.316655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:14.316655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:14.316655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:14.316655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:14.316655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:14.316655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:14.316655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:14.316655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:14.316655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:14.316655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:14.316655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:14.316655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:25:14.316655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:25:14.316655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:25:14.317707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:25:14.317707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:14.317707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:14.317707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:14.317707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:14.317707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:14.317707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:14.317707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:14.317707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:14.317707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:14.317707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:14.317707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:14.317707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:14.317707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:14.317707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:14.317707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:14.317707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:14.318725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:25:14.318725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:25:14.318725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:25:14.318725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:25:14.318725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:25:14.318725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:25:14.318725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:14.318725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:14.321400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 426/832] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:14.322352 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc157be-d38b-40ad-afb4-c5ad5690ac19 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:14.332356 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b07c1d-b6df-4407-9171-28822b8a0c82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:25:14.337513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-91b07c1d-b6df-4407-9171-28822b8a0c82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:25:14.337513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:14.337513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:14.337513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:14.337513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:14.337513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:14.337513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:14.337513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:14.337513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:14.337513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:14.337513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:14.337513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:14.337513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:25:14.337513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:25:14.337513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:25:14.337513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:25:14.338494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:25:14.338494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:25:14.338494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:25:14.338494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:25:14.338494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:14.338494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:14.338494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:14.338494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:14.338494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:14.338494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:14.338494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:14.338494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:14.338494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:14.338494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:14.338494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:14.338494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:14.339336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:14.339336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:14.339336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:14.339336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:14.339336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:25:14.339336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:25:14.339336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:25:14.339336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:25:14.339336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:25:14.339336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:25:14.339336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:25:14.339336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:14.339336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 407/833] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:25:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:14.340014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b07c1d-b6df-4407-9171-28822b8a0c82 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:14.389037 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:14.397457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:14.442759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc157be-d38b-40ad-afb4-c5ad5690ac19 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:14.455400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b07c1d-b6df-4407-9171-28822b8a0c82 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:14.478406 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48da52-eb10-415e-a5b5-2a92a62381ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:14.478996 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48da52-eb10-415e-a5b5-2a92a62381ed None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:14.479273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48da52-eb10-415e-a5b5-2a92a62381ed None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:14.479540 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48da52-eb10-415e-a5b5-2a92a62381ed None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:14.480245 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48da52-eb10-415e-a5b5-2a92a62381ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:14.480544 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48da52-eb10-415e-a5b5-2a92a62381ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:14.481967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48da52-eb10-415e-a5b5-2a92a62381ed None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:14.495204 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 427/834] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:25:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:14.496290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48da52-eb10-415e-a5b5-2a92a62381ed None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:14.502357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c54073-e0d5-4c73-aee1-5881db181d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:14.502967 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c54073-e0d5-4c73-aee1-5881db181d9a None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:14.503238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c54073-e0d5-4c73-aee1-5881db181d9a None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:14.503496 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c54073-e0d5-4c73-aee1-5881db181d9a None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:14.504612 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c54073-e0d5-4c73-aee1-5881db181d9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:14.504919 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c54073-e0d5-4c73-aee1-5881db181d9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:14.507238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c54073-e0d5-4c73-aee1-5881db181d9a None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:14.525941 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 408/835] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:25:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:14.526937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c54073-e0d5-4c73-aee1-5881db181d9a None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:14.604293 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48da52-eb10-415e-a5b5-2a92a62381ed None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:14.619403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:14.624848 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:14.632484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c54073-e0d5-4c73-aee1-5881db181d9a None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:14.639773 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ba23d-00dc-4582-b975-e7a8b70d547f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:14.640393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ba23d-00dc-4582-b975-e7a8b70d547f None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:14.640666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ba23d-00dc-4582-b975-e7a8b70d547f None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:14.640929 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ba23d-00dc-4582-b975-e7a8b70d547f None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:14.641573 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ba23d-00dc-4582-b975-e7a8b70d547f None admin] RBAC: Adding query filter params (name=admin) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:14.641872 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ba23d-00dc-4582-b975-e7a8b70d547f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:14.643232 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ba23d-00dc-4582-b975-e7a8b70d547f None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:14.652285 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 428/836] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:25:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:14.653046 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ba23d-00dc-4582-b975-e7a8b70d547f None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:14.671325 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f2139-a4e3-445d-ac14-a9791d0dc69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:14.671952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f2139-a4e3-445d-ac14-a9791d0dc69c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:14.672376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f2139-a4e3-445d-ac14-a9791d0dc69c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:14.672603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f2139-a4e3-445d-ac14-a9791d0dc69c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:14.673179 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f2139-a4e3-445d-ac14-a9791d0dc69c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:14.674537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f2139-a4e3-445d-ac14-a9791d0dc69c None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:14.684720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-379f2139-a4e3-445d-ac14-a9791d0dc69c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 22:25:14.684720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:14.684720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:14.684720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:14.684720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:14.684720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:14.684720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:14.684720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:14.684720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 22:25:14.684720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:14.684720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:14.684720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:14.684720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:14.684720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:14.684720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:14.684720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:14.684720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:14.685779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:14.685779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:14.685779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:14.685779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:14.685779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:14.685779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:14.685779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:14.685779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:14.685779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:14.685779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:14.685779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:14.685779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:14.685779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:14.685779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:14.685779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:14.685779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:14.688492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:14.688492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:14.688492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:14.688492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:14.688492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:14.688492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:14.688492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:14.688492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:14.688492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:14.688492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:14.688492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:14.688492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:14.688492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 22:25:14.688492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:14.688492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 409/837] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:25:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:14.689279 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f2139-a4e3-445d-ac14-a9791d0dc69c None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:14.762761 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ba23d-00dc-4582-b975-e7a8b70d547f None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:14.792660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f2139-a4e3-445d-ac14-a9791d0dc69c None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:14.798894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998ff9f-fffd-47de-abfb-1de7e79caed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:14.799555 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998ff9f-fffd-47de-abfb-1de7e79caed5 None admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:14.799829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998ff9f-fffd-47de-abfb-1de7e79caed5 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:14.800098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998ff9f-fffd-47de-abfb-1de7e79caed5 None admin] PATH_INFO: `/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/980157efae8f43b59dc063fda04c0348/roles/f91580fb2e5840bcb4f4054732e2e796` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:14.817126 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998ff9f-fffd-47de-abfb-1de7e79caed5 None admin] RBAC: Authorizing `identity:create_grant(project_id=7799820fc7c64fd6a99a0a9ccf1338e3, user_id=980157efae8f43b59dc063fda04c0348, role_id=f91580fb2e5840bcb4f4054732e2e796)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:14.818562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998ff9f-fffd-47de-abfb-1de7e79caed5 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:14.833027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff2c57-e2dc-42a3-b87a-2fd4c128ae20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:14.833625 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff2c57-e2dc-42a3-b87a-2fd4c128ae20 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:14.833900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff2c57-e2dc-42a3-b87a-2fd4c128ae20 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:14.834165 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff2c57-e2dc-42a3-b87a-2fd4c128ae20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:14.835603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff2c57-e2dc-42a3-b87a-2fd4c128ae20 None admin] RBAC: Adding query filter params (role.id=a836108447dc439e994611bc74d53d9c, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=7f21105679f245a19ea4850e6c010dbf) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:14.835905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff2c57-e2dc-42a3-b87a-2fd4c128ae20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:14.837374 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff2c57-e2dc-42a3-b87a-2fd4c128ae20 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:14.843236 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 410/838] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:25:14 2022] GET /identity/v3/role_assignments?user.id=7f21105679f245a19ea4850e6c010dbf&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=a836108447dc439e994611bc74d53d9c => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:14.844092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff2c57-e2dc-42a3-b87a-2fd4c128ae20 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:14.871324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:14.896174 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 429/839] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:25:14 2022] PUT /identity/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/980157efae8f43b59dc063fda04c0348/roles/f91580fb2e5840bcb4f4054732e2e796 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:25:14.896174 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998ff9f-fffd-47de-abfb-1de7e79caed5 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:14.947970 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:15.006325 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff2c57-e2dc-42a3-b87a-2fd4c128ae20 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:15.058871 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39213636-6c3c-4370-a21e-c2c4b3081711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:15.058871 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39213636-6c3c-4370-a21e-c2c4b3081711 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:15.058871 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39213636-6c3c-4370-a21e-c2c4b3081711 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:15.058871 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39213636-6c3c-4370-a21e-c2c4b3081711 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:15.059265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998ff9f-fffd-47de-abfb-1de7e79caed5 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:15.070816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39213636-6c3c-4370-a21e-c2c4b3081711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:15.072464 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39213636-6c3c-4370-a21e-c2c4b3081711 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:15.089502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-39213636-6c3c-4370-a21e-c2c4b3081711 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:15.089502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:15.089502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:15.089502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:15.089502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:15.089502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:15.089502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:15.089502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:15.089502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:15.089502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:15.089502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:15.089502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:15.089502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:15.089502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:15.089502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:15.089502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:15.090494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:15.090494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:15.090494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:15.090494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:15.090494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:15.090494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:15.090494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:15.090494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:15.090494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:15.090494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:15.090494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:15.090494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:15.090494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:15.090494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:15.090494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:15.090494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:15.090494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:15.091462 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:15.091462 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:15.091462 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:15.091462 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 411/840] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:15.097306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39213636-6c3c-4370-a21e-c2c4b3081711 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:15.102450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20636024-461b-4196-bc32-6a502e7955ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:15.103974 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20636024-461b-4196-bc32-6a502e7955ba None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:15.104286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20636024-461b-4196-bc32-6a502e7955ba None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:15.104548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20636024-461b-4196-bc32-6a502e7955ba None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:15.105196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20636024-461b-4196-bc32-6a502e7955ba None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:15.105495 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20636024-461b-4196-bc32-6a502e7955ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:15.106870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20636024-461b-4196-bc32-6a502e7955ba None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:15.117174 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 430/841] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:25:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:15.124700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20636024-461b-4196-bc32-6a502e7955ba None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:15.197476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:15.227881 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39213636-6c3c-4370-a21e-c2c4b3081711 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:15.235134 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:15.245674 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20636024-461b-4196-bc32-6a502e7955ba None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:15.266688 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a1e66-847d-4c3b-b2ed-77f0cdb8ba93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:15.267342 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a1e66-847d-4c3b-b2ed-77f0cdb8ba93 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:15.267622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a1e66-847d-4c3b-b2ed-77f0cdb8ba93 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:15.267954 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a1e66-847d-4c3b-b2ed-77f0cdb8ba93 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:15.268657 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a1e66-847d-4c3b-b2ed-77f0cdb8ba93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:15.269060 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a1e66-847d-4c3b-b2ed-77f0cdb8ba93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:15.270433 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a1e66-847d-4c3b-b2ed-77f0cdb8ba93 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:15.282756 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8c9ae-0084-46c0-84e9-ee63f49cfb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:15.283357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8c9ae-0084-46c0-84e9-ee63f49cfb08 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:15.283676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8c9ae-0084-46c0-84e9-ee63f49cfb08 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:15.283945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8c9ae-0084-46c0-84e9-ee63f49cfb08 None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:15.284616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e8c9ae-0084-46c0-84e9-ee63f49cfb08 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:15.284725 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 412/842] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:15.284907 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e8c9ae-0084-46c0-84e9-ee63f49cfb08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:15.286247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e8c9ae-0084-46c0-84e9-ee63f49cfb08 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:15.293117 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a1e66-847d-4c3b-b2ed-77f0cdb8ba93 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:15.295041 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 431/843] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:25:15 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:15.414510 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a1e66-847d-4c3b-b2ed-77f0cdb8ba93 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:15.459449 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ad2cc-b4fb-4a82-8828-ef759d770a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:15.460112 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ad2cc-b4fb-4a82-8828-ef759d770a36 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:15.460413 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ad2cc-b4fb-4a82-8828-ef759d770a36 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:15.460674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ad2cc-b4fb-4a82-8828-ef759d770a36 None admin] PATH_INFO: `/v3/users/swift` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:15.462909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:15.471355 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ad2cc-b4fb-4a82-8828-ef759d770a36 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:15.473491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ad2cc-b4fb-4a82-8828-ef759d770a36 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:15.486907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-531ad2cc-b4fb-4a82-8828-ef759d770a36 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:25:15.486907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:15.486907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:15.486907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:15.486907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:15.486907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:15.486907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:15.486907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:15.486907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:15.486907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:15.486907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:15.486907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:15.486907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:25:15.486907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:25:15.486907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:25:15.486907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:25:15.487914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:15.487914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:15.487914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:25:15.487914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:15.487914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:25:15.487914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:15.487914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:15.487914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:15.487914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:15.487914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:15.487914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:15.487914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:15.487914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:15.487914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:15.487914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:15.487914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:15.487914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:15.487914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:15.489280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:15.489280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:15.489280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:25:15.489280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:25:15.489280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:25:15.489280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:25:15.489280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:25:15.489280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:25:15.489280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 22:25:15.489280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:15.489280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 413/844] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:25:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:15.495142 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8c9ae-0084-46c0-84e9-ee63f49cfb08 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:15.605730 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8c9ae-0084-46c0-84e9-ee63f49cfb08 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:15.643530 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76b9fa-3e2e-452f-9e91-d9e81d5ff2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:15.644541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76b9fa-3e2e-452f-9e91-d9e81d5ff2c7 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:15.645026 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76b9fa-3e2e-452f-9e91-d9e81d5ff2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:15.645486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76b9fa-3e2e-452f-9e91-d9e81d5ff2c7 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:15.646392 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76b9fa-3e2e-452f-9e91-d9e81d5ff2c7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:15.646872 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76b9fa-3e2e-452f-9e91-d9e81d5ff2c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:15.649075 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76b9fa-3e2e-452f-9e91-d9e81d5ff2c7 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:15.670839 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 432/845] 10.222.0.59 () {56 vars in 1135 bytes} [Mon Aug 29 22:25:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:15.676089 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ad2cc-b4fb-4a82-8828-ef759d770a36 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:15.784792 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ad2cc-b4fb-4a82-8828-ef759d770a36 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:15.822050 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5bf7eb-91e8-4471-ab1c-f03418d08ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:15.822664 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5bf7eb-91e8-4471-ab1c-f03418d08ed8 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:15.822934 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5bf7eb-91e8-4471-ab1c-f03418d08ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:15.823197 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5bf7eb-91e8-4471-ab1c-f03418d08ed8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:15.888158 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:15.898234 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5bf7eb-91e8-4471-ab1c-f03418d08ed8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:15.900009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5bf7eb-91e8-4471-ab1c-f03418d08ed8 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:16.009562 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 414/846] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:25:15 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:25:16.015209 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76b9fa-3e2e-452f-9e91-d9e81d5ff2c7 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:16.031678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:16.142759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76b9fa-3e2e-452f-9e91-d9e81d5ff2c7 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:16.183053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a4973-19cb-482a-bd46-3c36346e2876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:16.183725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a4973-19cb-482a-bd46-3c36346e2876 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:16.183989 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a4973-19cb-482a-bd46-3c36346e2876 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:16.184282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a4973-19cb-482a-bd46-3c36346e2876 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:16.196044 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a4973-19cb-482a-bd46-3c36346e2876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:16.199986 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a4973-19cb-482a-bd46-3c36346e2876 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:16.214265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-ff8a4973-19cb-482a-bd46-3c36346e2876 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:16.214265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:16.214265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:16.214265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:16.214265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:16.214265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:16.214265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:16.214265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:16.214265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:16.214265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:16.214265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:16.214265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:16.214265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:16.214265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:16.214265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:16.214265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:16.215330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:16.215330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:16.215330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:16.215330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:16.215330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:16.215330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:16.215330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:16.215330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:16.215330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:16.215330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:16.215330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:16.215330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:16.215330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:16.215330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:16.215330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:16.215330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:16.215330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:16.216354 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:16.216354 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:16.216354 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:16.216354 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 433/847] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:16.222685 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5bf7eb-91e8-4471-ab1c-f03418d08ed8 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:16.322818 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:16.329902 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5bf7eb-91e8-4471-ab1c-f03418d08ed8 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:16.370947 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f238450-c387-4e4b-96d9-58873f19f430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:16.371936 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f238450-c387-4e4b-96d9-58873f19f430 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:16.372444 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f238450-c387-4e4b-96d9-58873f19f430 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:16.372922 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f238450-c387-4e4b-96d9-58873f19f430 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:16.373760 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f238450-c387-4e4b-96d9-58873f19f430 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:16.374361 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f238450-c387-4e4b-96d9-58873f19f430 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:16.375974 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f238450-c387-4e4b-96d9-58873f19f430 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:16.389667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 415/848] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:16.396201 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a4973-19cb-482a-bd46-3c36346e2876 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:16.464975 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:16.505962 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a4973-19cb-482a-bd46-3c36346e2876 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:16.545680 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3591f-d914-42ed-8c0c-8a24e30fcb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:16.546321 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3591f-d914-42ed-8c0c-8a24e30fcb4c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:16.546611 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3591f-d914-42ed-8c0c-8a24e30fcb4c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:16.546890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3591f-d914-42ed-8c0c-8a24e30fcb4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:16.555337 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3591f-d914-42ed-8c0c-8a24e30fcb4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:16.556985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3591f-d914-42ed-8c0c-8a24e30fcb4c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:16.571395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-49c3591f-d914-42ed-8c0c-8a24e30fcb4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:16.571395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:16.571395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:16.571395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:16.571395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:16.571395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:16.571395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:16.571395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:16.571395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:16.571395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:16.571395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:16.571395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:16.571395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:25:16.571395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:25:16.571395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:25:16.572394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:25:16.572394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:16.572394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:16.572394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:16.572394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:16.572394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:16.572394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:16.572394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:16.572394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:16.572394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:16.572394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:16.572394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:16.572394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:16.572394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:16.572394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:16.572394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:16.572394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:16.573319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:25:16.573319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:25:16.573319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:25:16.573319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:25:16.573319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:25:16.573319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:25:16.573319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:16.573319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:16.573319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 434/849] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:16.579562 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f238450-c387-4e4b-96d9-58873f19f430 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:16.690610 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f238450-c387-4e4b-96d9-58873f19f430 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:16.728518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395a001-501d-4ba0-a01e-ee2cbd34ba24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:16.729649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395a001-501d-4ba0-a01e-ee2cbd34ba24 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:16.730133 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395a001-501d-4ba0-a01e-ee2cbd34ba24 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:16.730607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395a001-501d-4ba0-a01e-ee2cbd34ba24 None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:16.731519 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395a001-501d-4ba0-a01e-ee2cbd34ba24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:16.732100 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395a001-501d-4ba0-a01e-ee2cbd34ba24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:16.733792 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395a001-501d-4ba0-a01e-ee2cbd34ba24 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:16.748376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 416/850] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:25:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:16.756180 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3591f-d914-42ed-8c0c-8a24e30fcb4c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:16.876894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3591f-d914-42ed-8c0c-8a24e30fcb4c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:16.904326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:16.916980 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4040c-f4f9-477d-b364-397f3bcc9693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:16.917636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4040c-f4f9-477d-b364-397f3bcc9693 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:16.920419 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4040c-f4f9-477d-b364-397f3bcc9693 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:16.920419 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4040c-f4f9-477d-b364-397f3bcc9693 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:16.920419 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4040c-f4f9-477d-b364-397f3bcc9693 None admin] RBAC: Adding query filter params (role.id=a836108447dc439e994611bc74d53d9c, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=f1f0503ae1b24ce6a8adb3cfd7973f14) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:16.920419 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4040c-f4f9-477d-b364-397f3bcc9693 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:16.921252 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4040c-f4f9-477d-b364-397f3bcc9693 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:16.927945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 435/851] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:25:16 2022] GET /identity/v3/role_assignments?user.id=f1f0503ae1b24ce6a8adb3cfd7973f14&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=a836108447dc439e994611bc74d53d9c => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:16.968122 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2956cb-e15e-44bb-8e2c-01a7d3f60e9c None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:16.968447 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2956cb-e15e-44bb-8e2c-01a7d3f60e9c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:16.968681 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2956cb-e15e-44bb-8e2c-01a7d3f60e9c None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:16.970269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 417/852] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:16.976185 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a8699-27ab-4fba-a1a1-38cdcc2fa6aa None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:16.976795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a8699-27ab-4fba-a1a1-38cdcc2fa6aa None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:16.977268 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a8699-27ab-4fba-a1a1-38cdcc2fa6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:16.997968 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-f10a8699-27ab-4fba-a1a1-38cdcc2fa6aa None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:16.999537 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f10a8699-27ab-4fba-a1a1-38cdcc2fa6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:17.062912 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9b9f5-becc-43ab-a21a-8285ae56adb9 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:17.063237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9b9f5-becc-43ab-a21a-8285ae56adb9 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:17.063453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9b9f5-becc-43ab-a21a-8285ae56adb9 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:17.065279 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 418/853] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:17.071629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027f30e-517a-426d-a43d-0e4efd905612 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:17.071961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027f30e-517a-426d-a43d-0e4efd905612 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:17.072213 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027f30e-517a-426d-a43d-0e4efd905612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:17.092207 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-e027f30e-517a-426d-a43d-0e4efd905612 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:17.092500 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 436/854] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:17.093470 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-e027f30e-517a-426d-a43d-0e4efd905612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:17.104551 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:17.170755 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 419/855] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:17.372972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a8699-27ab-4fba-a1a1-38cdcc2fa6aa None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:17.452966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027f30e-517a-426d-a43d-0e4efd905612 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:17.490848 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a8699-27ab-4fba-a1a1-38cdcc2fa6aa None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:17.514056 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:17.540193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0eded-6df8-4435-9dc7-cc2f8dfb9946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:17.540838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0eded-6df8-4435-9dc7-cc2f8dfb9946 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:17.554459 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0eded-6df8-4435-9dc7-cc2f8dfb9946 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:17.554459 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0eded-6df8-4435-9dc7-cc2f8dfb9946 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:17.554459 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0eded-6df8-4435-9dc7-cc2f8dfb9946 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:25:17.557385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-6cc0eded-6df8-4435-9dc7-cc2f8dfb9946 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:25:17.557385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:17.557385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:17.557385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:17.557385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:17.557385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:17.557385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:17.557385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:17.557385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:17.557385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:17.557385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:17.557385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:17.557385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:25:17.557385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:25:17.557385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:25:17.557385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:25:17.558390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:25:17.558390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:25:17.558390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:25:17.558390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:25:17.558390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:17.558390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:17.558390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:17.558390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:17.558390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:17.558390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:17.558390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:17.558390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:17.558390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:17.558390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:17.558390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:17.558390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:17.559208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:17.559208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:17.559208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:17.559208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:17.559208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:25:17.559208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:25:17.559208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:25:17.559208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:25:17.559208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:25:17.559208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:25:17.559208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:25:17.559208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:17.559208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 437/856] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:25:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:17.560524 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bac7f-6d4c-4d7c-935a-1d0844a8db99 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:17.560803 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bac7f-6d4c-4d7c-935a-1d0844a8db99 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:17.561074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bac7f-6d4c-4d7c-935a-1d0844a8db99 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:17.566394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 438/857] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:17.567533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b9cd2-7aaa-4c1f-979e-e1ee4376915f None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:17.567819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b9cd2-7aaa-4c1f-979e-e1ee4376915f None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:17.568089 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b9cd2-7aaa-4c1f-979e-e1ee4376915f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:17.587201 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027f30e-517a-426d-a43d-0e4efd905612 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:17.588751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-6f5b9cd2-7aaa-4c1f-979e-e1ee4376915f None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:17.588751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6f5b9cd2-7aaa-4c1f-979e-e1ee4376915f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:17.634376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051f1ff-f781-42d0-ab93-f0c61c89fff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:17.635145 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051f1ff-f781-42d0-ab93-f0c61c89fff0 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:17.635428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051f1ff-f781-42d0-ab93-f0c61c89fff0 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:17.635740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051f1ff-f781-42d0-ab93-f0c61c89fff0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:17.636318 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051f1ff-f781-42d0-ab93-f0c61c89fff0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:17.637704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051f1ff-f781-42d0-ab93-f0c61c89fff0 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:17.647616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-a051f1ff-f781-42d0-ab93-f0c61c89fff0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 22:25:17.647616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:17.647616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:17.647616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:17.647616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:17.647616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:17.647616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:17.647616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:17.647616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 22:25:17.647616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:17.647616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:17.647616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:17.647616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:17.647616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:17.647616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:17.647616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:17.648727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:17.648727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:17.648727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:17.648727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:17.648727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:17.648727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:17.648727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:17.648727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:17.648727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:17.648727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:17.648727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:17.648727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:17.648727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:17.648727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:17.648727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:17.648727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:17.648727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:17.649689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:17.649689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:17.649689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:17.649689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:17.649689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:17.649689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:17.649689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:17.649689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:17.649689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:17.649689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:17.649689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:17.649689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:17.649689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 22:25:17.649689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:17.649689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 420/858] 10.222.0.59 () {56 vars in 1132 bytes} [Mon Aug 29 22:25:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:17.650633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051f1ff-f781-42d0-ab93-f0c61c89fff0 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:17.667539 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:17.684188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 439/859] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:17.684188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b9cd2-7aaa-4c1f-979e-e1ee4376915f None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:17.796199 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b9cd2-7aaa-4c1f-979e-e1ee4376915f None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:17.806683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:17.835766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebae079-4ee6-44fb-9221-b7cc61098260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:17.836531 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebae079-4ee6-44fb-9221-b7cc61098260 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:17.836751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebae079-4ee6-44fb-9221-b7cc61098260 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:17.837014 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebae079-4ee6-44fb-9221-b7cc61098260 None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:17.837667 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebae079-4ee6-44fb-9221-b7cc61098260 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:17.838470 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebae079-4ee6-44fb-9221-b7cc61098260 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:17.840062 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebae079-4ee6-44fb-9221-b7cc61098260 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:17.847110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051f1ff-f781-42d0-ab93-f0c61c89fff0 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:17.850403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 440/860] 10.222.0.59 () {56 vars in 1126 bytes} [Mon Aug 29 22:25:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:17.850756 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebae079-4ee6-44fb-9221-b7cc61098260 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:17.923299 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1324b36c-b872-4292-9a62-256c6a965a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:17.924169 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324b36c-b872-4292-9a62-256c6a965a61 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:17.924664 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324b36c-b872-4292-9a62-256c6a965a61 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:17.925175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324b36c-b872-4292-9a62-256c6a965a61 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:17.929493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1324b36c-b872-4292-9a62-256c6a965a61 None admin] RBAC: Adding query filter params (name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:17.936656 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1324b36c-b872-4292-9a62-256c6a965a61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:17.938952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1324b36c-b872-4292-9a62-256c6a965a61 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:17.969088 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 421/861] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:25:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:17.970427 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1324b36c-b872-4292-9a62-256c6a965a61 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:17.977641 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebae079-4ee6-44fb-9221-b7cc61098260 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:18.021177 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a86a64-6110-4a02-93e4-3207d64306c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:18.021788 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a86a64-6110-4a02-93e4-3207d64306c6 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:18.022100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a86a64-6110-4a02-93e4-3207d64306c6 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:18.022547 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a86a64-6110-4a02-93e4-3207d64306c6 None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:18.023282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a86a64-6110-4a02-93e4-3207d64306c6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:18.023964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a86a64-6110-4a02-93e4-3207d64306c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:18.025402 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a86a64-6110-4a02-93e4-3207d64306c6 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:18.044590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 441/862] 10.222.0.59 () {56 vars in 1126 bytes} [Mon Aug 29 22:25:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:18.045870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a86a64-6110-4a02-93e4-3207d64306c6 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:18.106909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:18.171012 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1324b36c-b872-4292-9a62-256c6a965a61 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:18.176436 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:18.221054 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d451a-2e18-4f94-a5bb-240af6e02672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:18.221861 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d451a-2e18-4f94-a5bb-240af6e02672 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:18.222244 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d451a-2e18-4f94-a5bb-240af6e02672 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:18.222646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d451a-2e18-4f94-a5bb-240af6e02672 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:18.240292 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d451a-2e18-4f94-a5bb-240af6e02672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:18.242128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d451a-2e18-4f94-a5bb-240af6e02672 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:18.260350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-a47d451a-2e18-4f94-a5bb-240af6e02672 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:18.260350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:18.260350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:18.260350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:18.260350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:18.260350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:18.260350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:18.260350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:18.260350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:18.260350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:18.260350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:18.260350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:18.260350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:18.260350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:18.260350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:18.260350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:18.261398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:18.261398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:18.261398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:18.261398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:18.261398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:18.261398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:18.261398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:18.261398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:18.261398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:18.261398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:18.261398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:18.261398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:18.261398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:18.261398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:18.261398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:18.261398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:18.261398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:18.262366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:18.262366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:18.262366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:18.266901 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 422/863] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:18.271168 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d451a-2e18-4f94-a5bb-240af6e02672 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:18.312323 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a86a64-6110-4a02-93e4-3207d64306c6 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:18.350656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685e14d-82b8-4fa8-8cd3-70612b5b9d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:18.351574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685e14d-82b8-4fa8-8cd3-70612b5b9d0f None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:18.351853 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685e14d-82b8-4fa8-8cd3-70612b5b9d0f None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:18.352320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685e14d-82b8-4fa8-8cd3-70612b5b9d0f None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:18.352861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685e14d-82b8-4fa8-8cd3-70612b5b9d0f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:18.354282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685e14d-82b8-4fa8-8cd3-70612b5b9d0f None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:18.423604 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d451a-2e18-4f94-a5bb-240af6e02672 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:18.436306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 442/864] 10.222.0.59 () {60 vars in 1141 bytes} [Mon Aug 29 22:25:18 2022] POST /identity/v3/services => generated 251 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:25:18.465804 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:18.488674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9f6d7-9c6c-412a-91bc-6a3c021ac5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:18.489409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9f6d7-9c6c-412a-91bc-6a3c021ac5ed None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:18.489683 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9f6d7-9c6c-412a-91bc-6a3c021ac5ed None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:18.489946 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9f6d7-9c6c-412a-91bc-6a3c021ac5ed None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:18.490625 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf9f6d7-9c6c-412a-91bc-6a3c021ac5ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:18.491484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf9f6d7-9c6c-412a-91bc-6a3c021ac5ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:18.493246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf9f6d7-9c6c-412a-91bc-6a3c021ac5ed None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:18.508696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 423/865] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:18.514351 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685e14d-82b8-4fa8-8cd3-70612b5b9d0f None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:18.636822 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:18.644219 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685e14d-82b8-4fa8-8cd3-70612b5b9d0f None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:18.682737 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acfbed-0483-496d-b701-f75163a92ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:18.683379 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acfbed-0483-496d-b701-f75163a92ee1 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:18.683793 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acfbed-0483-496d-b701-f75163a92ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:18.684261 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acfbed-0483-496d-b701-f75163a92ee1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:18.693748 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26acfbed-0483-496d-b701-f75163a92ee1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:18.695336 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26acfbed-0483-496d-b701-f75163a92ee1 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:18.714338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-26acfbed-0483-496d-b701-f75163a92ee1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:25:18.714338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:18.714338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:18.714338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:18.714338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:18.714338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:18.714338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:18.714338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:18.714338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:18.714338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:18.714338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:18.714338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:18.714338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:25:18.714338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:25:18.714338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:25:18.714338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:25:18.715377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:18.715377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:18.715377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:25:18.715377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:18.715377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:25:18.715377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:18.715377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:18.715377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:18.715377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:18.715377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:18.715377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:18.715377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:18.715377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:18.715377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:18.715377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:18.715377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:18.715377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:18.715377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:18.716519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:18.716519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:18.716519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:25:18.716519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:25:18.716519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:25:18.716519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:25:18.716519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:25:18.716519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:25:18.716519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:25:18.716519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:18.717683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 443/866] 10.222.0.59 () {56 vars in 1117 bytes} [Mon Aug 29 22:25:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:18.723730 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9f6d7-9c6c-412a-91bc-6a3c021ac5ed None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:18.890270 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9f6d7-9c6c-412a-91bc-6a3c021ac5ed None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:18.930376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075609a-862b-4514-8ef6-57987648e6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:18.931349 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075609a-862b-4514-8ef6-57987648e6bb None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:18.931721 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075609a-862b-4514-8ef6-57987648e6bb None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:18.932000 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075609a-862b-4514-8ef6-57987648e6bb None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:18.932857 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075609a-862b-4514-8ef6-57987648e6bb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:18.933102 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075609a-862b-4514-8ef6-57987648e6bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:18.934595 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075609a-862b-4514-8ef6-57987648e6bb None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:18.963319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 424/867] 10.222.0.59 () {56 vars in 1149 bytes} [Mon Aug 29 22:25:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:18.969419 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acfbed-0483-496d-b701-f75163a92ee1 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:19.083879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acfbed-0483-496d-b701-f75163a92ee1 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:19.123113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5088399-d61d-4e2b-bae3-9720e79d447c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:19.124209 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5088399-d61d-4e2b-bae3-9720e79d447c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:19.124727 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5088399-d61d-4e2b-bae3-9720e79d447c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:19.125211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5088399-d61d-4e2b-bae3-9720e79d447c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:19.152608 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06fd60-9744-4803-b60e-cdfc8e8678a4 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:19.152937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06fd60-9744-4803-b60e-cdfc8e8678a4 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:19.153145 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06fd60-9744-4803-b60e-cdfc8e8678a4 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:19.153992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:19.154831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 425/868] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:19.160099 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623dc412-b3fa-4f40-8581-ce8d772f1d3e None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:19.160407 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623dc412-b3fa-4f40-8581-ce8d772f1d3e None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:19.160676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623dc412-b3fa-4f40-8581-ce8d772f1d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:19.180548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-623dc412-b3fa-4f40-8581-ce8d772f1d3e None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:19.181826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-623dc412-b3fa-4f40-8581-ce8d772f1d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:19.207934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5088399-d61d-4e2b-bae3-9720e79d447c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:19.212102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5088399-d61d-4e2b-bae3-9720e79d447c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:19.218349 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:19.273385 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 426/869] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:19 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:19.332273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 444/870] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:25:18 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:25:19.334405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623dc412-b3fa-4f40-8581-ce8d772f1d3e None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:19.447847 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623dc412-b3fa-4f40-8581-ce8d772f1d3e None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:19.489989 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d728e-3b48-4865-873c-f66d9631b589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:19.491092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d728e-3b48-4865-873c-f66d9631b589 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:19.491419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d728e-3b48-4865-873c-f66d9631b589 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:19.491685 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d728e-3b48-4865-873c-f66d9631b589 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:19.505380 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d728e-3b48-4865-873c-f66d9631b589 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:19.508723 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d728e-3b48-4865-873c-f66d9631b589 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:19.526863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-e57d728e-3b48-4865-873c-f66d9631b589 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:19.526863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:19.526863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:19.526863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:19.526863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:19.526863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:19.526863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:19.526863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:19.526863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:19.526863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:19.526863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:19.526863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:19.526863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:19.526863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:19.526863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:19.526863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:19.527905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:19.527905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:19.527905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:19.527905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:19.527905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:19.527905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:19.527905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:19.527905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:19.527905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:19.527905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:19.527905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:19.527905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:19.527905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:19.527905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:19.527905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:19.527905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:19.527905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:19.528838 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:19.528838 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:19.528838 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:19.530199 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 427/871] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:19.536068 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5088399-d61d-4e2b-bae3-9720e79d447c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:19.567733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d728e-3b48-4865-873c-f66d9631b589 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:19.596429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:19.600064 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:19.657129 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5088399-d61d-4e2b-bae3-9720e79d447c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:19.717094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e039ab3-8727-4c3c-8138-73ede0894655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:19.718220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d728e-3b48-4865-873c-f66d9631b589 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:19.718753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e039ab3-8727-4c3c-8138-73ede0894655 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:19.719467 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e039ab3-8727-4c3c-8138-73ede0894655 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:19.720490 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e039ab3-8727-4c3c-8138-73ede0894655 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:19.723092 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e039ab3-8727-4c3c-8138-73ede0894655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:19.723869 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e039ab3-8727-4c3c-8138-73ede0894655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:19.726688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e039ab3-8727-4c3c-8138-73ede0894655 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:19.752019 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 445/872] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:19.758437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e039ab3-8727-4c3c-8138-73ede0894655 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:19.767606 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc863dd-2061-4e3c-b3b2-f0db52f185ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:19.768335 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc863dd-2061-4e3c-b3b2-f0db52f185ef None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:19.768576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc863dd-2061-4e3c-b3b2-f0db52f185ef None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:19.768834 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc863dd-2061-4e3c-b3b2-f0db52f185ef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:19.769446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc863dd-2061-4e3c-b3b2-f0db52f185ef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:19.770881 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc863dd-2061-4e3c-b3b2-f0db52f185ef None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:19.782390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-0fc863dd-2061-4e3c-b3b2-f0db52f185ef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:25:19.782390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:19.782390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:19.782390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:19.782390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:19.782390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:19.782390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:19.782390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:19.782390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:25:19.782390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:19.782390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:19.782390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:19.782390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:19.782390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:19.782390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:19.782390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:19.782390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:19.783455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:19.783455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:19.783455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:19.783455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:19.783455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:19.783455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:19.783455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:19.783455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:19.783455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:19.783455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:19.783455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:19.783455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:19.783455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:19.783455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:19.783455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:19.783455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:19.784367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:19.784367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:19.784367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:19.784367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:19.784367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:19.784367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:19.784367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:19.784367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:19.784367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:19.784367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:19.784367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:19.784367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:19.784367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:25:19.784367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:19.789406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 428/873] 10.222.0.59 () {56 vars in 1126 bytes} [Mon Aug 29 22:25:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:19.790486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc863dd-2061-4e3c-b3b2-f0db52f185ef None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:19.949950 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:19.951230 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e039ab3-8727-4c3c-8138-73ede0894655 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:19.978874 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:19.988792 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc863dd-2061-4e3c-b3b2-f0db52f185ef None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:20.015010 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21687f58-594d-40f3-a0a8-898dbe6ca7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:20.015558 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21687f58-594d-40f3-a0a8-898dbe6ca7e7 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:20.015836 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21687f58-594d-40f3-a0a8-898dbe6ca7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:20.016105 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21687f58-594d-40f3-a0a8-898dbe6ca7e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:20.028435 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21687f58-594d-40f3-a0a8-898dbe6ca7e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:20.029889 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21687f58-594d-40f3-a0a8-898dbe6ca7e7 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:20.046344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-21687f58-594d-40f3-a0a8-898dbe6ca7e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:20.046344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:20.046344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:20.046344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:20.046344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:20.046344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:20.046344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:20.046344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:20.046344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:20.046344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:20.046344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:20.046344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:20.046344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:25:20.046344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:25:20.046344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:25:20.048039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:25:20.048039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:20.048039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:20.048039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:20.048039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:20.048039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:20.048039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:20.048039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:20.048039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:20.048039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:20.048039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:20.048039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:20.048039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:20.048039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:20.048039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:20.048039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:20.048039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:20.049690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:25:20.049690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:25:20.049690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:25:20.049690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:25:20.049690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:25:20.049690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:25:20.049690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:20.049690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:20.049690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 446/874] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:20.059792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21687f58-594d-40f3-a0a8-898dbe6ca7e7 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:20.067321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8002a02-54e3-4b8d-aa6a-d6721e28123b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:20.067321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8002a02-54e3-4b8d-aa6a-d6721e28123b None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:20.067321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8002a02-54e3-4b8d-aa6a-d6721e28123b None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:20.067321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8002a02-54e3-4b8d-aa6a-d6721e28123b None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:20.069698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8002a02-54e3-4b8d-aa6a-d6721e28123b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:20.069698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8002a02-54e3-4b8d-aa6a-d6721e28123b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:20.069698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8002a02-54e3-4b8d-aa6a-d6721e28123b None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:20.080537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 429/875] 10.222.0.59 () {56 vars in 1122 bytes} [Mon Aug 29 22:25:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:20.094831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8002a02-54e3-4b8d-aa6a-d6721e28123b None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:20.245923 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21687f58-594d-40f3-a0a8-898dbe6ca7e7 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:20.288363 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8002a02-54e3-4b8d-aa6a-d6721e28123b None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:20.298026 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc935c5-ac90-42ad-aa07-2c8482e029d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:20.298831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc935c5-ac90-42ad-aa07-2c8482e029d5 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:20.299182 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc935c5-ac90-42ad-aa07-2c8482e029d5 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:20.299527 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc935c5-ac90-42ad-aa07-2c8482e029d5 None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:20.300337 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc935c5-ac90-42ad-aa07-2c8482e029d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:20.300711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc935c5-ac90-42ad-aa07-2c8482e029d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:20.302247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc935c5-ac90-42ad-aa07-2c8482e029d5 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:20.317049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 447/876] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:25:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:20.325557 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc935c5-ac90-42ad-aa07-2c8482e029d5 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:20.331181 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8a6e7-a646-4b08-aa88-4aabc30df0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:20.332771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8a6e7-a646-4b08-aa88-4aabc30df0ff None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:20.333050 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8a6e7-a646-4b08-aa88-4aabc30df0ff None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:20.333327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8a6e7-a646-4b08-aa88-4aabc30df0ff None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:20.333966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8a6e7-a646-4b08-aa88-4aabc30df0ff None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:20.334303 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8a6e7-a646-4b08-aa88-4aabc30df0ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:20.336155 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8a6e7-a646-4b08-aa88-4aabc30df0ff None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:20.342234 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 430/877] 10.222.0.59 () {56 vars in 1122 bytes} [Mon Aug 29 22:25:20 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:20.389779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:20.472864 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc935c5-ac90-42ad-aa07-2c8482e029d5 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:20.524741 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2965a-9885-4be0-a714-7b564a78e651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:20.525360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2965a-9885-4be0-a714-7b564a78e651 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:20.525634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2965a-9885-4be0-a714-7b564a78e651 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:20.525902 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2965a-9885-4be0-a714-7b564a78e651 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:20.526640 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2965a-9885-4be0-a714-7b564a78e651 None admin] RBAC: Adding query filter params (role.id=f91580fb2e5840bcb4f4054732e2e796, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=980157efae8f43b59dc063fda04c0348) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:20.526941 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2965a-9885-4be0-a714-7b564a78e651 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:20.528486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2965a-9885-4be0-a714-7b564a78e651 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:20.535834 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 448/878] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:25:20 2022] GET /identity/v3/role_assignments?user.id=980157efae8f43b59dc063fda04c0348&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=f91580fb2e5840bcb4f4054732e2e796 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:20.550967 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5a52d-aa10-4d34-952e-55ed7f11b975 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:20.551571 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5a52d-aa10-4d34-952e-55ed7f11b975 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:20.552183 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5a52d-aa10-4d34-952e-55ed7f11b975 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:20.555653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 431/879] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:20.562451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472851be-68f2-4d15-a361-01851c5fa79c None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:20.563066 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472851be-68f2-4d15-a361-01851c5fa79c None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:20.563651 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472851be-68f2-4d15-a361-01851c5fa79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:20.588518 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-472851be-68f2-4d15-a361-01851c5fa79c None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:20.589768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-472851be-68f2-4d15-a361-01851c5fa79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:20.681905 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 449/880] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:20.824289 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573d584-f598-45d3-bc9f-601c463877a0 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:20.825033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573d584-f598-45d3-bc9f-601c463877a0 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:20.825033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573d584-f598-45d3-bc9f-601c463877a0 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:20.827111 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 432/881] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:20.832650 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596b489-a41c-49fa-9893-1f709b8be92f None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:20.832997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596b489-a41c-49fa-9893-1f709b8be92f None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:20.833272 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596b489-a41c-49fa-9893-1f709b8be92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:20.854468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-d596b489-a41c-49fa-9893-1f709b8be92f None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:20.855683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-d596b489-a41c-49fa-9893-1f709b8be92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:20.886017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:20.941977 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 450/882] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:21.033573 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573d584-f598-45d3-bc9f-601c463877a0 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:21.054087 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:21.169325 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573d584-f598-45d3-bc9f-601c463877a0 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:21.207171 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ebba5-ee14-4abe-9e51-f851cb3d66ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:21.207791 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ebba5-ee14-4abe-9e51-f851cb3d66ce None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:21.208186 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ebba5-ee14-4abe-9e51-f851cb3d66ce None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:21.208465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ebba5-ee14-4abe-9e51-f851cb3d66ce None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:21.209821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ebba5-ee14-4abe-9e51-f851cb3d66ce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:21.211306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ebba5-ee14-4abe-9e51-f851cb3d66ce None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:21.246931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596b489-a41c-49fa-9893-1f709b8be92f None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:21.324523 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 433/883] 10.222.0.59 () {60 vars in 1141 bytes} [Mon Aug 29 22:25:21 2022] POST /identity/v3/services => generated 269 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:25:21.397610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596b489-a41c-49fa-9893-1f709b8be92f None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:21.451704 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24abf4-f057-46c4-996c-b2c267323cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:21.452958 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24abf4-f057-46c4-996c-b2c267323cae None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:21.453562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24abf4-f057-46c4-996c-b2c267323cae None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:21.454008 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24abf4-f057-46c4-996c-b2c267323cae None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:21.454774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b24abf4-f057-46c4-996c-b2c267323cae None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:21.456610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b24abf4-f057-46c4-996c-b2c267323cae None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:21.468377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-5b24abf4-f057-46c4-996c-b2c267323cae None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 22:25:21.468377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:21.468377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:21.468377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:21.468377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:21.468377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:21.468377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:21.468377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:21.468377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 22:25:21.468377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:21.468377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:21.468377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:21.468377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:21.468377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:21.468377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:21.468377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:21.468377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:21.469677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:21.469677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:21.469677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:21.469677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:21.469677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:21.469677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:21.469677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:21.469677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:21.469677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:21.469677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:21.469677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:21.469677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:21.469677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:21.469677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:21.469677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:21.469677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:21.470711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:21.470711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:21.470711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:21.470711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:21.470711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:21.470711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:21.470711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:21.470711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:21.470711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:21.470711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:21.470711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:21.470711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:21.470711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 22:25:21.470711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:21.473283 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 451/884] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:25:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:21.480662 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ebba5-ee14-4abe-9e51-f851cb3d66ce None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:21.567972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:21.611920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ebba5-ee14-4abe-9e51-f851cb3d66ce None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:21.656911 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafdea73-f3b0-4e89-ae0a-43e0c8149a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:21.657522 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafdea73-f3b0-4e89-ae0a-43e0c8149a31 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:21.657827 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafdea73-f3b0-4e89-ae0a-43e0c8149a31 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:21.658154 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafdea73-f3b0-4e89-ae0a-43e0c8149a31 None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:21.659546 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafdea73-f3b0-4e89-ae0a-43e0c8149a31 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:21.659941 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafdea73-f3b0-4e89-ae0a-43e0c8149a31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:21.661760 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafdea73-f3b0-4e89-ae0a-43e0c8149a31 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:21.668337 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 434/885] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:25:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:21.674212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24abf4-f057-46c4-996c-b2c267323cae None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:21.702678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:21.822024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24abf4-f057-46c4-996c-b2c267323cae None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:21.859679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263e87b-820e-4c20-8aeb-3d9a309dbccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:21.860352 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263e87b-820e-4c20-8aeb-3d9a309dbccf None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:21.860632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263e87b-820e-4c20-8aeb-3d9a309dbccf None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:21.860899 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263e87b-820e-4c20-8aeb-3d9a309dbccf None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:21.861534 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263e87b-820e-4c20-8aeb-3d9a309dbccf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:21.861830 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263e87b-820e-4c20-8aeb-3d9a309dbccf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:21.863194 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263e87b-820e-4c20-8aeb-3d9a309dbccf None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:21.870787 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 452/886] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:25:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:21.876400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafdea73-f3b0-4e89-ae0a-43e0c8149a31 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:22.064781 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafdea73-f3b0-4e89-ae0a-43e0c8149a31 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:22.108341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156ae90-2106-492f-8bae-b7bd163a72a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:22.109388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156ae90-2106-492f-8bae-b7bd163a72a5 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:22.109890 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156ae90-2106-492f-8bae-b7bd163a72a5 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:22.110366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156ae90-2106-492f-8bae-b7bd163a72a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:22.111752 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156ae90-2106-492f-8bae-b7bd163a72a5 None admin] RBAC: Adding query filter params (interface=public, service_id=0471629f929e408389050736702d2a07, region_id=RegionOne) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:22.112499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156ae90-2106-492f-8bae-b7bd163a72a5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:22.114805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156ae90-2106-492f-8bae-b7bd163a72a5 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:22.119064 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:22.127478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 435/887] 10.222.0.59 () {56 vars in 1251 bytes} [Mon Aug 29 22:25:21 2022] GET /identity/v3/endpoints?service_id=0471629f929e408389050736702d2a07&interface=public®ion_id=RegionOne => generated 194 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:22.136489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263e87b-820e-4c20-8aeb-3d9a309dbccf None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:22.259258 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:22.283270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263e87b-820e-4c20-8aeb-3d9a309dbccf None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:22.325546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc47537-035b-44e8-8225-249ce4c129d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:22.326265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc47537-035b-44e8-8225-249ce4c129d0 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:22.326615 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc47537-035b-44e8-8225-249ce4c129d0 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:22.327001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc47537-035b-44e8-8225-249ce4c129d0 None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:22.327779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc47537-035b-44e8-8225-249ce4c129d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:22.329587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc47537-035b-44e8-8225-249ce4c129d0 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:22.337438 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 453/888] 10.222.0.59 () {56 vars in 1087 bytes} [Mon Aug 29 22:25:22 2022] GET /identity/v3/services => generated 1268 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:25:22.543090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512452e-11d6-4dc7-aba9-a23e1838acf3 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:22.543424 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512452e-11d6-4dc7-aba9-a23e1838acf3 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:22.543610 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512452e-11d6-4dc7-aba9-a23e1838acf3 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:22.545289 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 436/889] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:22.554191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bfc3e-2102-474a-9d2e-0e4c4a683432 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:22.554677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bfc3e-2102-474a-9d2e-0e4c4a683432 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:22.555130 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bfc3e-2102-474a-9d2e-0e4c4a683432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:22.582389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-5c0bfc3e-2102-474a-9d2e-0e4c4a683432 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:22.583503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-5c0bfc3e-2102-474a-9d2e-0e4c4a683432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:22.665900 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 454/890] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:23.001112 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7476796f-af54-4243-9f5f-3f492740ec97 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:23.001778 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7476796f-af54-4243-9f5f-3f492740ec97 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:23.001778 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7476796f-af54-4243-9f5f-3f492740ec97 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:23.003213 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 437/891] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:23.009845 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c5339-9107-4ca8-9354-8014d273c411 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:23.010136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c5339-9107-4ca8-9354-8014d273c411 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:23.010391 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c5339-9107-4ca8-9354-8014d273c411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:23.019764 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7476796f-af54-4243-9f5f-3f492740ec97 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:23.036269 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-9b7c5339-9107-4ca8-9354-8014d273c411 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:23.038616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-9b7c5339-9107-4ca8-9354-8014d273c411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:23.140550 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7476796f-af54-4243-9f5f-3f492740ec97 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:23.145992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 455/892] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:23.178785 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990a77a-346b-4045-afc8-24c97e73fdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:23.179320 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990a77a-346b-4045-afc8-24c97e73fdc0 None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:23.179595 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990a77a-346b-4045-afc8-24c97e73fdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:23.179868 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990a77a-346b-4045-afc8-24c97e73fdc0 None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:23.180576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990a77a-346b-4045-afc8-24c97e73fdc0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:23.181948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990a77a-346b-4045-afc8-24c97e73fdc0 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:23.264989 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 438/893] 10.222.0.59 () {60 vars in 1141 bytes} [Mon Aug 29 22:25:23 2022] POST /identity/v3/services => generated 241 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:25:23.458145 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c5339-9107-4ca8-9354-8014d273c411 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:23.470632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:23.587346 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c5339-9107-4ca8-9354-8014d273c411 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:23.623631 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df742b8-5bec-4b5d-a680-64d1e6b878e5 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:23.623956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df742b8-5bec-4b5d-a680-64d1e6b878e5 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:23.624211 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df742b8-5bec-4b5d-a680-64d1e6b878e5 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:23.625809 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 439/894] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:25:23.631047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f4466-4977-4109-a47d-4e38a4d3ae06 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:23.631343 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f4466-4977-4109-a47d-4e38a4d3ae06 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:23.631600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f4466-4977-4109-a47d-4e38a4d3ae06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:23.634857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c303f-0889-4557-9f91-8aa041310284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:23.636038 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c303f-0889-4557-9f91-8aa041310284 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:23.636659 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c303f-0889-4557-9f91-8aa041310284 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:23.636956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c303f-0889-4557-9f91-8aa041310284 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:23.648439 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c303f-0889-4557-9f91-8aa041310284 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:25:23.650545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-996f4466-4977-4109-a47d-4e38a4d3ae06 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:23.651862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-996f4466-4977-4109-a47d-4e38a4d3ae06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:23.653463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-e99c303f-0889-4557-9f91-8aa041310284 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:25:23.653463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:23.653463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:23.653463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:23.653463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:23.653463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:23.653463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:23.653463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:23.653463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:23.653463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:23.653463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:23.653463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:23.653463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:25:23.653463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:25:23.653463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:25:23.653463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:25:23.654851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:25:23.654851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:25:23.654851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:25:23.654851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:25:23.654851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:23.654851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:23.654851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:23.654851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:23.654851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:23.654851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:23.654851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:23.654851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:23.654851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:23.654851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:23.654851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:23.654851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:23.656115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:23.656115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:23.656115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:23.656115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:23.656115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:25:23.656115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:25:23.656115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:25:23.656115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:25:23.656115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:25:23.656115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:25:23.656115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:25:23.656115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:23.656115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 456/895] 10.222.0.59 () {56 vars in 1120 bytes} [Mon Aug 29 22:25:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:23.666725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c303f-0889-4557-9f91-8aa041310284 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:23.716354 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:23.734945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 440/896] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:23.777995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:23.823473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c303f-0889-4557-9f91-8aa041310284 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:23.861186 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac4670-8098-4445-a4d4-dc6c7a79fb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:23.861801 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac4670-8098-4445-a4d4-dc6c7a79fb21 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:23.862078 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac4670-8098-4445-a4d4-dc6c7a79fb21 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:23.862346 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac4670-8098-4445-a4d4-dc6c7a79fb21 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:23.863011 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac4670-8098-4445-a4d4-dc6c7a79fb21 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:23.863306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac4670-8098-4445-a4d4-dc6c7a79fb21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:23.864804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac4670-8098-4445-a4d4-dc6c7a79fb21 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:23.875171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 457/897] 10.222.0.59 () {56 vars in 1115 bytes} [Mon Aug 29 22:25:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:23.881554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f4466-4977-4109-a47d-4e38a4d3ae06 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:23.993578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f4466-4977-4109-a47d-4e38a4d3ae06 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:24.030603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1abab6-45e1-4366-904a-02c823f2f289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:24.031253 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1abab6-45e1-4366-904a-02c823f2f289 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:24.031494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1abab6-45e1-4366-904a-02c823f2f289 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:24.031759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1abab6-45e1-4366-904a-02c823f2f289 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:24.044582 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1abab6-45e1-4366-904a-02c823f2f289 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:24.046029 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1abab6-45e1-4366-904a-02c823f2f289 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:24.062439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-3b1abab6-45e1-4366-904a-02c823f2f289 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:24.062439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:24.062439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:24.062439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:24.062439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:24.062439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:24.062439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:24.062439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:24.062439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:24.062439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:24.062439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:24.062439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:24.062439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:24.062439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:24.062439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:24.062439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:24.063453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:24.063453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:24.063453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:24.063453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:24.063453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:24.063453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:24.063453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:24.063453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:24.063453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:24.063453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:24.063453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:24.063453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:24.063453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:24.063453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:24.063453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:24.063453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:24.063453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:24.064486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:24.064486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:24.064486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:24.064813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac4670-8098-4445-a4d4-dc6c7a79fb21 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:24.064912 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 441/898] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:24.071789 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1abab6-45e1-4366-904a-02c823f2f289 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:24.123342 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:24.187920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1abab6-45e1-4366-904a-02c823f2f289 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:24.211392 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac4670-8098-4445-a4d4-dc6c7a79fb21 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:24.233408 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b516af-38a4-4902-a135-fe556c23745a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:24.234301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b516af-38a4-4902-a135-fe556c23745a None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:24.234734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b516af-38a4-4902-a135-fe556c23745a None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:24.235145 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b516af-38a4-4902-a135-fe556c23745a None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:24.236496 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b516af-38a4-4902-a135-fe556c23745a None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:24.237108 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b516af-38a4-4902-a135-fe556c23745a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:24.239051 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b516af-38a4-4902-a135-fe556c23745a None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:24.242732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:24.254352 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48ca7b-7850-4439-a7b1-acbd9ecba062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:24.254352 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48ca7b-7850-4439-a7b1-acbd9ecba062 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:24.254352 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48ca7b-7850-4439-a7b1-acbd9ecba062 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:24.254352 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48ca7b-7850-4439-a7b1-acbd9ecba062 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:24.254778 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48ca7b-7850-4439-a7b1-acbd9ecba062 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:24.255722 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48ca7b-7850-4439-a7b1-acbd9ecba062 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:24.263001 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 442/899] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:24.265732 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b516af-38a4-4902-a135-fe556c23745a None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:24.270924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-dd48ca7b-7850-4439-a7b1-acbd9ecba062 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 22:25:24.270924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:24.270924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:24.270924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:24.270924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:24.270924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:24.270924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:24.270924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:24.270924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 22:25:24.270924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:24.270924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:24.270924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:24.270924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:24.270924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:24.270924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:24.270924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:24.273626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:24.273626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:24.273626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:24.273626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:24.273626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:24.273626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:24.273626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:24.273626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:24.273626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:24.273626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:24.273626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:24.273626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:24.273626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:24.273626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:24.273626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:24.273626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:24.273626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:24.274844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:24.274844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:24.274844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:24.274844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:24.274844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:24.274844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:24.274844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:24.274844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:24.274844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:24.274844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:24.274844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:24.274844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:24.274844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 22:25:24.274844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:24.280493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 458/900] 10.222.0.59 () {56 vars in 1132 bytes} [Mon Aug 29 22:25:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:24.281319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48ca7b-7850-4439-a7b1-acbd9ecba062 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:24.391647 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:24.395112 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b516af-38a4-4902-a135-fe556c23745a None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:24.409462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48ca7b-7850-4439-a7b1-acbd9ecba062 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:24.435661 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39019a89-053f-4268-a207-f6b679e24d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:24.436330 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39019a89-053f-4268-a207-f6b679e24d83 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:24.436605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39019a89-053f-4268-a207-f6b679e24d83 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:24.436870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39019a89-053f-4268-a207-f6b679e24d83 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:24.447074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6f1ab-f910-4dbb-8c14-5c28b77cbcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:24.447691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6f1ab-f910-4dbb-8c14-5c28b77cbcc1 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:24.447964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6f1ab-f910-4dbb-8c14-5c28b77cbcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:24.448277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6f1ab-f910-4dbb-8c14-5c28b77cbcc1 None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:24.449626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6f1ab-f910-4dbb-8c14-5c28b77cbcc1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:24.450034 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6f1ab-f910-4dbb-8c14-5c28b77cbcc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:24.451795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6f1ab-f910-4dbb-8c14-5c28b77cbcc1 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:24.454143 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39019a89-053f-4268-a207-f6b679e24d83 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:24.455645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39019a89-053f-4268-a207-f6b679e24d83 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:24.457157 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 459/901] 10.222.0.59 () {56 vars in 1126 bytes} [Mon Aug 29 22:25:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:24.459228 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157c146-c6ab-41fc-afa9-01182d77277a None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:24.460505 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157c146-c6ab-41fc-afa9-01182d77277a None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:24.460693 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157c146-c6ab-41fc-afa9-01182d77277a None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:24.462315 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 460/902] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:24.465044 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157c146-c6ab-41fc-afa9-01182d77277a None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:24.476428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-39019a89-053f-4268-a207-f6b679e24d83 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:25:24.476428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:24.476428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:24.476428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:24.476428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:24.476428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:24.476428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:24.476428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:24.476428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:24.476428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:24.476428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:24.476428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:24.476428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:25:24.476428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:25:24.476428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:25:24.476428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:25:24.477476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:24.477476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:24.477476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:25:24.477476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:24.477476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:25:24.477476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:24.477476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:24.477476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:24.477476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:24.477476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:24.477476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:24.477476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:24.477476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:24.477476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:24.477476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:24.477476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:24.477476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:24.477476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:24.483768 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:24.483768 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:24.483768 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:25:24.483768 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:25:24.483768 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:25:24.483768 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:25:24.483768 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:25:24.483768 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:25:24.483768 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:25:24.483768 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:24.483768 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 443/903] 10.222.0.59 () {56 vars in 1117 bytes} [Mon Aug 29 22:25:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:24.483768 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97561e0d-1133-45bc-94ae-d36af5e1ba63 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:24.483768 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97561e0d-1133-45bc-94ae-d36af5e1ba63 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:24.484533 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97561e0d-1133-45bc-94ae-d36af5e1ba63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:24.505870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-97561e0d-1133-45bc-94ae-d36af5e1ba63 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:24.507358 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-97561e0d-1133-45bc-94ae-d36af5e1ba63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:24.541533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:24.614073 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157c146-c6ab-41fc-afa9-01182d77277a None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:24.619629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 444/904] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:24.619629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97561e0d-1133-45bc-94ae-d36af5e1ba63 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:24.666722 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817eb413-ce28-4921-9020-6c19b8263b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:24.667740 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817eb413-ce28-4921-9020-6c19b8263b4f None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:24.668247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817eb413-ce28-4921-9020-6c19b8263b4f None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:24.668725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817eb413-ce28-4921-9020-6c19b8263b4f None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:24.669556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817eb413-ce28-4921-9020-6c19b8263b4f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:24.670058 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817eb413-ce28-4921-9020-6c19b8263b4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:24.671731 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817eb413-ce28-4921-9020-6c19b8263b4f None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:24.677862 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 461/905] 10.222.0.59 () {56 vars in 1126 bytes} [Mon Aug 29 22:25:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:24.683729 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817eb413-ce28-4921-9020-6c19b8263b4f None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:24.735057 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:24.752284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97561e0d-1133-45bc-94ae-d36af5e1ba63 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:24.799422 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d2c13-4f6c-42f9-abdb-2a16b09b7614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:24.800575 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d2c13-4f6c-42f9-abdb-2a16b09b7614 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:24.801072 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d2c13-4f6c-42f9-abdb-2a16b09b7614 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:24.801553 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d2c13-4f6c-42f9-abdb-2a16b09b7614 None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:24.802510 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d2c13-4f6c-42f9-abdb-2a16b09b7614 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:24.803723 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d2c13-4f6c-42f9-abdb-2a16b09b7614 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:24.806463 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d2c13-4f6c-42f9-abdb-2a16b09b7614 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:24.809986 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817eb413-ce28-4921-9020-6c19b8263b4f None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:24.830209 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:24.830938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 445/906] 10.222.0.59 () {56 vars in 1149 bytes} [Mon Aug 29 22:25:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:24.839033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d2c13-4f6c-42f9-abdb-2a16b09b7614 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:24.851407 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb955491-68a9-41e5-a8ec-2491e70d5782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:24.852299 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb955491-68a9-41e5-a8ec-2491e70d5782 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:24.852706 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb955491-68a9-41e5-a8ec-2491e70d5782 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:24.853378 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb955491-68a9-41e5-a8ec-2491e70d5782 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:24.854360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb955491-68a9-41e5-a8ec-2491e70d5782 None admin] RBAC: Adding query filter params (interface=public, service_id=a4eb16afa15942ae809a17b154c2ddd9, region_id=RegionOne) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:24.854670 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb955491-68a9-41e5-a8ec-2491e70d5782 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:24.856410 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb955491-68a9-41e5-a8ec-2491e70d5782 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:24.863750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 462/907] 10.222.0.59 () {56 vars in 1251 bytes} [Mon Aug 29 22:25:24 2022] GET /identity/v3/endpoints?service_id=a4eb16afa15942ae809a17b154c2ddd9&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:24.873506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb955491-68a9-41e5-a8ec-2491e70d5782 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:24.978149 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d2c13-4f6c-42f9-abdb-2a16b09b7614 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:25.020342 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb955491-68a9-41e5-a8ec-2491e70d5782 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:25.025231 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ce622-9664-4d54-bb85-9b6f5079f2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:25.025861 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ce622-9664-4d54-bb85-9b6f5079f2e7 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:25.026158 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ce622-9664-4d54-bb85-9b6f5079f2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:25.026426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ce622-9664-4d54-bb85-9b6f5079f2e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:25.048825 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:25.063522 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03421b02-483f-436d-9095-00386e467614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:25.063522 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03421b02-483f-436d-9095-00386e467614 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:25.063522 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03421b02-483f-436d-9095-00386e467614 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:25.063522 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03421b02-483f-436d-9095-00386e467614 None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:25.063943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03421b02-483f-436d-9095-00386e467614 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:25.063943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03421b02-483f-436d-9095-00386e467614 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:25.078315 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 463/908] 10.222.0.59 () {56 vars in 1087 bytes} [Mon Aug 29 22:25:24 2022] GET /identity/v3/services => generated 1497 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:25:25.078315 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03421b02-483f-436d-9095-00386e467614 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:25.123248 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ce622-9664-4d54-bb85-9b6f5079f2e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:25.124987 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ce622-9664-4d54-bb85-9b6f5079f2e7 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:25.151591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:25.238259 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03421b02-483f-436d-9095-00386e467614 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:25.269690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 446/909] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:25:24 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 22:25:25.276069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ce622-9664-4d54-bb85-9b6f5079f2e7 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:25.291933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f3f52-9e60-4160-b714-6da6bbb987f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:25.292599 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f3f52-9e60-4160-b714-6da6bbb987f7 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:25.292936 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f3f52-9e60-4160-b714-6da6bbb987f7 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:25.293313 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f3f52-9e60-4160-b714-6da6bbb987f7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:25.293868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f3f52-9e60-4160-b714-6da6bbb987f7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:25.295293 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f3f52-9e60-4160-b714-6da6bbb987f7 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:25.305718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-145f3f52-9e60-4160-b714-6da6bbb987f7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 22:25:25.305718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:25.305718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:25.305718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:25.305718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:25.305718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:25.305718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:25.305718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:25.305718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 22:25:25.305718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:25.305718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:25.305718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:25.305718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:25.305718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:25.305718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:25.305718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:25.305718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:25.307365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:25.307365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:25.307365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:25.307365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:25.307365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:25.307365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:25.307365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:25.307365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:25.307365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:25.307365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:25.307365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:25.307365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:25.307365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:25.307365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:25.307365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:25.307365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:25.308635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:25.308635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:25.308635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:25.308635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:25.308635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:25.308635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:25.308635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:25.308635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:25.308635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:25.308635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:25.308635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:25.308635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:25.308635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 22:25:25.308635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:25.308635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 464/910] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:25:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:25.313008 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f3f52-9e60-4160-b714-6da6bbb987f7 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:25.347308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:25.413217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ce622-9664-4d54-bb85-9b6f5079f2e7 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:25.457409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf38af-e891-4364-bb69-4c40bcf47fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:25.458052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf38af-e891-4364-bb69-4c40bcf47fb3 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:25.458417 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf38af-e891-4364-bb69-4c40bcf47fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:25.458751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf38af-e891-4364-bb69-4c40bcf47fb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:25.460610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f3f52-9e60-4160-b714-6da6bbb987f7 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:25.472562 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf38af-e891-4364-bb69-4c40bcf47fb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:25.473834 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf38af-e891-4364-bb69-4c40bcf47fb3 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:25.485604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:25.497212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-56bf38af-e891-4364-bb69-4c40bcf47fb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:25.497212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:25.497212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:25.497212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:25.497212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:25.497212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:25.497212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:25.497212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:25.497212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:25.497212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:25.497212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:25.497212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:25.497212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:25.497212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:25.497212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:25.497212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:25.499566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:25.499566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:25.499566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:25.499566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:25.499566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:25.499566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:25.499566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:25.499566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:25.499566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:25.499566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:25.499566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:25.499566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:25.499566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:25.499566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:25.499566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:25.499566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:25.499566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:25.500887 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:25.500887 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:25.500887 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:25.503322 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 447/911] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:25.504911 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf38af-e891-4364-bb69-4c40bcf47fb3 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:25.525209 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddfb26-05a7-417b-a826-6c415eea0b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:25.526646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddfb26-05a7-417b-a826-6c415eea0b9c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:25.527163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddfb26-05a7-417b-a826-6c415eea0b9c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:25.527435 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddfb26-05a7-417b-a826-6c415eea0b9c None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:25.528072 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ddfb26-05a7-417b-a826-6c415eea0b9c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:25.528478 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ddfb26-05a7-417b-a826-6c415eea0b9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:25.529884 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ddfb26-05a7-417b-a826-6c415eea0b9c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:25.535323 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 465/912] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:25:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:25.536643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f5bd7-a0c7-4953-b0a6-fcd832865735 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:25.536921 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f5bd7-a0c7-4953-b0a6-fcd832865735 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:25.537737 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f5bd7-a0c7-4953-b0a6-fcd832865735 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:25.539227 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 466/913] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:25.541998 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f5bd7-a0c7-4953-b0a6-fcd832865735 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:25.636078 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:25.640112 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf38af-e891-4364-bb69-4c40bcf47fb3 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:25.679812 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f5bd7-a0c7-4953-b0a6-fcd832865735 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:25.681764 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca3616-18df-4e85-b69d-a6d5435153cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:25.682381 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca3616-18df-4e85-b69d-a6d5435153cc None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:25.682657 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca3616-18df-4e85-b69d-a6d5435153cc None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:25.682922 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca3616-18df-4e85-b69d-a6d5435153cc None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:25.683569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca3616-18df-4e85-b69d-a6d5435153cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:25.683862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca3616-18df-4e85-b69d-a6d5435153cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:25.685261 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca3616-18df-4e85-b69d-a6d5435153cc None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:25.703984 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 448/914] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:25.705174 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f5448-4fa0-480b-aeea-30db3cbdde51 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:25.705459 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f5448-4fa0-480b-aeea-30db3cbdde51 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:25.705734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f5448-4fa0-480b-aeea-30db3cbdde51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:25.719848 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a46a52-c475-4e9b-a1ce-1a32cf950299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:25.720744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a46a52-c475-4e9b-a1ce-1a32cf950299 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:25.721019 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a46a52-c475-4e9b-a1ce-1a32cf950299 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:25.721289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a46a52-c475-4e9b-a1ce-1a32cf950299 None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:25.721921 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a46a52-c475-4e9b-a1ce-1a32cf950299 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:25.722223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a46a52-c475-4e9b-a1ce-1a32cf950299 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:25.723645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a46a52-c475-4e9b-a1ce-1a32cf950299 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:25.727727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-6f0f5448-4fa0-480b-aeea-30db3cbdde51 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:25.728729 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6f0f5448-4fa0-480b-aeea-30db3cbdde51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:25.736306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 467/915] 10.222.0.59 () {56 vars in 1114 bytes} [Mon Aug 29 22:25:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:25.736306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a46a52-c475-4e9b-a1ce-1a32cf950299 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:25.781744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:25.816183 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 449/916] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:25.816183 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f5448-4fa0-480b-aeea-30db3cbdde51 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:25.861546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a46a52-c475-4e9b-a1ce-1a32cf950299 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:25.903636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed7137-0f88-42e5-8583-b3f3b915159c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:25.904213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed7137-0f88-42e5-8583-b3f3b915159c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:25.904500 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed7137-0f88-42e5-8583-b3f3b915159c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:25.904705 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed7137-0f88-42e5-8583-b3f3b915159c None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:25.912437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed7137-0f88-42e5-8583-b3f3b915159c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:25.913872 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed7137-0f88-42e5-8583-b3f3b915159c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:25.934059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-e6ed7137-0f88-42e5-8583-b3f3b915159c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:25.934059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:25.934059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:25.934059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:25.934059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:25.934059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:25.934059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:25.934059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:25.934059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:25.934059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:25.934059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:25.934059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:25.934059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:25:25.934059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:25:25.934059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:25:25.935102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:25:25.935102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:25.935102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:25.935102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:25.935102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:25.935102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:25.935102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:25.935102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:25.935102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:25.935102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:25.935102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:25.935102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:25.935102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:25.935102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:25.935102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:25.935102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:25.935102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:25.935985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:25:25.935985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:25:25.935985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:25:25.935985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:25:25.935985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:25:25.935985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:25:25.935985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:25.935985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:25.940634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 468/917] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:25.941357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed7137-0f88-42e5-8583-b3f3b915159c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:25.962115 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:25.977606 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f5448-4fa0-480b-aeea-30db3cbdde51 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:26.025035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4633c8-a476-4f89-b0de-5473a1e8fb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:26.026254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4633c8-a476-4f89-b0de-5473a1e8fb52 None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:26.027406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4633c8-a476-4f89-b0de-5473a1e8fb52 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:26.027914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4633c8-a476-4f89-b0de-5473a1e8fb52 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:26.029500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4633c8-a476-4f89-b0de-5473a1e8fb52 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:26.031680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4633c8-a476-4f89-b0de-5473a1e8fb52 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:26.067142 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:26.071929 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed7137-0f88-42e5-8583-b3f3b915159c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:26.106190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c3302-e915-4f20-997b-c6c01fb3f5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:26.106791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c3302-e915-4f20-997b-c6c01fb3f5e4 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:26.107063 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c3302-e915-4f20-997b-c6c01fb3f5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:26.107336 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c3302-e915-4f20-997b-c6c01fb3f5e4 None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:26.108108 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c3302-e915-4f20-997b-c6c01fb3f5e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:26.113511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c3302-e915-4f20-997b-c6c01fb3f5e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:26.113511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c3302-e915-4f20-997b-c6c01fb3f5e4 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:26.122879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 469/918] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:25:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:26.128819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c3302-e915-4f20-997b-c6c01fb3f5e4 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:26.159156 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 450/919] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:25:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:25:26.159156 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4633c8-a476-4f89-b0de-5473a1e8fb52 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:26.274191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c3302-e915-4f20-997b-c6c01fb3f5e4 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:26.299095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4633c8-a476-4f89-b0de-5473a1e8fb52 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:26.319841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafdaed8-00b6-4254-ae51-839b8bb3550c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:26.320493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafdaed8-00b6-4254-ae51-839b8bb3550c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:26.320773 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafdaed8-00b6-4254-ae51-839b8bb3550c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:26.321043 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafdaed8-00b6-4254-ae51-839b8bb3550c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:26.321818 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafdaed8-00b6-4254-ae51-839b8bb3550c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:26.323200 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafdaed8-00b6-4254-ae51-839b8bb3550c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:26.333860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-dafdaed8-00b6-4254-ae51-839b8bb3550c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:25:26.333860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:26.333860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:26.333860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:26.333860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:26.333860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:26.333860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:26.333860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:26.333860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:25:26.333860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:26.333860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:26.333860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:26.333860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:26.333860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:26.333860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:26.333860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:26.333860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:26.335282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:26.335282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:26.335282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:26.335282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:26.335282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:26.335282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:26.335282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:26.335282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:26.335282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:26.335282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:26.335282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:26.335282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:26.335282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:26.335282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:26.335282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:26.335282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:26.336360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:26.336360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:26.336360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:26.336360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:26.336360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:26.336360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:26.336360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:26.336360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:26.336360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:26.336360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:26.336360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:26.336360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:26.336360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:25:26.336360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:26.336360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 470/920] 10.222.0.59 () {56 vars in 1126 bytes} [Mon Aug 29 22:25:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:26.345475 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafdaed8-00b6-4254-ae51-839b8bb3550c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:26.348004 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb80a3-b005-4914-bc49-ac9ae444fd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:26.348004 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb80a3-b005-4914-bc49-ac9ae444fd0e None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:26.348004 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb80a3-b005-4914-bc49-ac9ae444fd0e None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:26.348004 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb80a3-b005-4914-bc49-ac9ae444fd0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:26.349566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb80a3-b005-4914-bc49-ac9ae444fd0e None admin] RBAC: Adding query filter params (role.id=04230301829e46deb6a9a0f33b868e35, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=980157efae8f43b59dc063fda04c0348) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:26.350534 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb80a3-b005-4914-bc49-ac9ae444fd0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:26.352243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb80a3-b005-4914-bc49-ac9ae444fd0e None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:26.359036 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 451/921] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:25:26 2022] GET /identity/v3/role_assignments?user.id=980157efae8f43b59dc063fda04c0348&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=04230301829e46deb6a9a0f33b868e35 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:26.380878 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:26.474041 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafdaed8-00b6-4254-ae51-839b8bb3550c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:26.516721 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f6ab8-18d5-4538-8485-a14d6a6f5d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:26.517327 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f6ab8-18d5-4538-8485-a14d6a6f5d9f None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:26.517660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f6ab8-18d5-4538-8485-a14d6a6f5d9f None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:26.517926 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f6ab8-18d5-4538-8485-a14d6a6f5d9f None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:26.518557 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f6ab8-18d5-4538-8485-a14d6a6f5d9f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:26.518851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f6ab8-18d5-4538-8485-a14d6a6f5d9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:26.520239 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f6ab8-18d5-4538-8485-a14d6a6f5d9f None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:26.526120 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 471/922] 10.222.0.59 () {56 vars in 1122 bytes} [Mon Aug 29 22:25:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:26.531822 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb80a3-b005-4914-bc49-ac9ae444fd0e None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:26.542028 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:26.640404 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb80a3-b005-4914-bc49-ac9ae444fd0e None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:26.675346 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f15a8-4652-4a8a-911c-c11044c8180e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:26.675956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f15a8-4652-4a8a-911c-c11044c8180e None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:26.676252 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f15a8-4652-4a8a-911c-c11044c8180e None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:26.676519 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f15a8-4652-4a8a-911c-c11044c8180e None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:26.677146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f15a8-4652-4a8a-911c-c11044c8180e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:26.677439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f15a8-4652-4a8a-911c-c11044c8180e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:26.679449 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f15a8-4652-4a8a-911c-c11044c8180e None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:26.685443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 452/923] 10.222.0.59 () {56 vars in 1122 bytes} [Mon Aug 29 22:25:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:26.691538 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f6ab8-18d5-4538-8485-a14d6a6f5d9f None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:26.788672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:26.804510 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f6ab8-18d5-4538-8485-a14d6a6f5d9f None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:26.842385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb9bb2-be0e-49b5-941c-18f20da86c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:26.843281 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb9bb2-be0e-49b5-941c-18f20da86c55 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:26.843733 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb9bb2-be0e-49b5-941c-18f20da86c55 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:26.844183 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb9bb2-be0e-49b5-941c-18f20da86c55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:26.845048 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb9bb2-be0e-49b5-941c-18f20da86c55 None admin] RBAC: Adding query filter params (interface=public, service_id=7442c744e74a4fccacdb5da724860d02, region_id=RegionOne) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:26.845499 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb9bb2-be0e-49b5-941c-18f20da86c55 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:26.847013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb9bb2-be0e-49b5-941c-18f20da86c55 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:26.853826 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 472/924] 10.222.0.59 () {56 vars in 1251 bytes} [Mon Aug 29 22:25:26 2022] GET /identity/v3/endpoints?service_id=7442c744e74a4fccacdb5da724860d02&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:26.861788 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f15a8-4652-4a8a-911c-c11044c8180e None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:26.948741 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:26.970499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f15a8-4652-4a8a-911c-c11044c8180e None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:27.007324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a4778-a3fb-4ba0-8a00-ca8a79e4a8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:27.007933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a4778-a3fb-4ba0-8a00-ca8a79e4a8cc None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:27.008226 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a4778-a3fb-4ba0-8a00-ca8a79e4a8cc None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:27.008536 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a4778-a3fb-4ba0-8a00-ca8a79e4a8cc None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:27.009492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a4778-a3fb-4ba0-8a00-ca8a79e4a8cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:27.011051 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a4778-a3fb-4ba0-8a00-ca8a79e4a8cc None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:27.017605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 453/925] 10.222.0.59 () {56 vars in 1087 bytes} [Mon Aug 29 22:25:26 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:25:27.203014 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57733248-c4c9-4c0f-ae7d-cd01db4de879 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:27.203295 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57733248-c4c9-4c0f-ae7d-cd01db4de879 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:27.203573 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57733248-c4c9-4c0f-ae7d-cd01db4de879 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:27.205191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 473/926] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:27.211231 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab099d-d999-4450-91fb-27f81dd8d022 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:27.211698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab099d-d999-4450-91fb-27f81dd8d022 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:27.211806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab099d-d999-4450-91fb-27f81dd8d022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:27.231371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-d1ab099d-d999-4450-91fb-27f81dd8d022 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:27.232600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d1ab099d-d999-4450-91fb-27f81dd8d022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:27.303699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 454/927] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:27.582148 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57733248-c4c9-4c0f-ae7d-cd01db4de879 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:27.690598 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57733248-c4c9-4c0f-ae7d-cd01db4de879 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:27.726764 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac50755-f048-45da-bb96-4120f8c38947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:27.727394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac50755-f048-45da-bb96-4120f8c38947 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:27.727668 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac50755-f048-45da-bb96-4120f8c38947 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:27.727936 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac50755-f048-45da-bb96-4120f8c38947 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:27.728545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac50755-f048-45da-bb96-4120f8c38947 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:27.729903 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac50755-f048-45da-bb96-4120f8c38947 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:27.738841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-0ac50755-f048-45da-bb96-4120f8c38947 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 22:25:27.738841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:27.738841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:27.738841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:27.738841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:27.738841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:27.738841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:27.738841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:27.738841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 22:25:27.738841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:27.738841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:27.738841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:27.738841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:27.738841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:27.738841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:27.738841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:27.739785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:27.739785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:27.739785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:27.739785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:27.739785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:27.739785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:27.739785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:27.739785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:27.739785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:27.739785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:27.739785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:27.739785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:27.739785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:27.739785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:27.739785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:27.739785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:27.739785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:27.741211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:27.741211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:27.741211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:27.741211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:27.741211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:27.741211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:27.741211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:27.741211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:27.741211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:27.741211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:27.741211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:27.741211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:27.741211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 22:25:27.741211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:27.741211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 474/928] 10.222.0.59 () {56 vars in 1132 bytes} [Mon Aug 29 22:25:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:27.746376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab099d-d999-4450-91fb-27f81dd8d022 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:27.826904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:27.854557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab099d-d999-4450-91fb-27f81dd8d022 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:27.891763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4ae11-9a7a-41a8-a6c6-331577a1bae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:27.892409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4ae11-9a7a-41a8-a6c6-331577a1bae8 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:27.892684 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4ae11-9a7a-41a8-a6c6-331577a1bae8 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:27.893019 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4ae11-9a7a-41a8-a6c6-331577a1bae8 None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:27.893652 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4ae11-9a7a-41a8-a6c6-331577a1bae8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:27.894000 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4ae11-9a7a-41a8-a6c6-331577a1bae8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:27.895463 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4ae11-9a7a-41a8-a6c6-331577a1bae8 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:27.902016 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 455/929] 10.222.0.59 () {56 vars in 1126 bytes} [Mon Aug 29 22:25:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:27.909461 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac50755-f048-45da-bb96-4120f8c38947 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:27.942414 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:28.064658 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac50755-f048-45da-bb96-4120f8c38947 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:28.126089 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b8502-2cd9-4593-a52f-4b1f41fd6b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:28.127020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b8502-2cd9-4593-a52f-4b1f41fd6b67 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:28.127472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b8502-2cd9-4593-a52f-4b1f41fd6b67 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:28.127916 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b8502-2cd9-4593-a52f-4b1f41fd6b67 None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:28.128994 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b8502-2cd9-4593-a52f-4b1f41fd6b67 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:28.129513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b8502-2cd9-4593-a52f-4b1f41fd6b67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:28.132562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b8502-2cd9-4593-a52f-4b1f41fd6b67 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:28.141968 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 475/930] 10.222.0.59 () {56 vars in 1126 bytes} [Mon Aug 29 22:25:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:28.149564 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4ae11-9a7a-41a8-a6c6-331577a1bae8 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:28.263880 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4ae11-9a7a-41a8-a6c6-331577a1bae8 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:28.296499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:28.303763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a0d14-a1f3-4709-afeb-185bd5c0eee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:28.304409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a0d14-a1f3-4709-afeb-185bd5c0eee1 None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:28.304689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a0d14-a1f3-4709-afeb-185bd5c0eee1 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:28.304955 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a0d14-a1f3-4709-afeb-185bd5c0eee1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:28.305631 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a0d14-a1f3-4709-afeb-185bd5c0eee1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:28.306988 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a0d14-a1f3-4709-afeb-185bd5c0eee1 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:28.316394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc2148-bc53-4030-96bc-7f2591f2503f None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:28.316683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc2148-bc53-4030-96bc-7f2591f2503f None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:28.316951 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc2148-bc53-4030-96bc-7f2591f2503f None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:28.319147 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 476/931] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:25:28.324753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b29e00-fa5b-4e05-9b21-96afa78b4eac None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:28.325037 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b29e00-fa5b-4e05-9b21-96afa78b4eac None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:28.325304 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b29e00-fa5b-4e05-9b21-96afa78b4eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:28.345161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-75b29e00-fa5b-4e05-9b21-96afa78b4eac None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:28.346483 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-75b29e00-fa5b-4e05-9b21-96afa78b4eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:28.421379 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 477/932] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:28.453061 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 456/933] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:25:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:25:28.602669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a6163-93d0-40d1-a9ab-780ebc849cc7 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:28.603198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a6163-93d0-40d1-a9ab-780ebc849cc7 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:28.603198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a6163-93d0-40d1-a9ab-780ebc849cc7 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:28.604886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 478/934] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:28.610140 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5396107-825d-443c-b5bc-16f17161658d None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:28.610741 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5396107-825d-443c-b5bc-16f17161658d None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:28.611018 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5396107-825d-443c-b5bc-16f17161658d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:28.632398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-c5396107-825d-443c-b5bc-16f17161658d None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:28.633606 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-c5396107-825d-443c-b5bc-16f17161658d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:28.705345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a6163-93d0-40d1-a9ab-780ebc849cc7 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:28.717249 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 457/935] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:28.718045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:28.816725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a6163-93d0-40d1-a9ab-780ebc849cc7 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:28.854774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7682ed9-7844-4759-b0c7-c393e6a6d94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:28.855387 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7682ed9-7844-4759-b0c7-c393e6a6d94e None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:28.855669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7682ed9-7844-4759-b0c7-c393e6a6d94e None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:28.855950 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7682ed9-7844-4759-b0c7-c393e6a6d94e None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:28.856686 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7682ed9-7844-4759-b0c7-c393e6a6d94e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:28.858137 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7682ed9-7844-4759-b0c7-c393e6a6d94e None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:28.860813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: WARNING keystone.server.flask.common [None req-b7682ed9-7844-4759-b0c7-c393e6a6d94e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 29 22:25:28.913689 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.sql.core [None req-b7682ed9-7844-4759-b0c7-c393e6a6d94e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 22:25:28.913689 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 22:25:28.913689 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [parameters: {'id': '929ed6dc6b4842f8806e23a456d2c60c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 22:25:28.913689 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=82165) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 22:25:28.924277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-b7682ed9-7844-4759-b0c7-c393e6a6d94e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 929ed6dc6b4842f8806e23a456d2c60c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 929ed6dc6b4842f8806e23a456d2c60c. Aug 29 22:25:28.924277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:28.924277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 22:25:28.924277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 22:25:28.924277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 22:25:28.924277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 22:25:28.924277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 22:25:28.924277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 22:25:28.924277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 22:25:28.924277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application conn.query(q) Aug 29 22:25:28.924277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 22:25:28.924277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 22:25:28.924277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 22:25:28.925274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application result.read() Aug 29 22:25:28.925274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 22:25:28.925274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 22:25:28.925274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 22:25:28.925274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 22:25:28.925274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 22:25:28.925274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 22:25:28.925274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 22:25:28.925274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 22:25:28.925274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 22:25:28.925274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:28.925274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 22:25:28.925274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:28.925274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:28.925274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 22:25:28.925274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 22:25:28.925274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 29 22:25:28.925274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 29 22:25:28.926468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 22:25:28.926468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application next(self.gen) Aug 29 22:25:28.926468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 22:25:28.926468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application yield resource Aug 29 22:25:28.926468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 22:25:28.926468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application next(self.gen) Aug 29 22:25:28.926468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 22:25:28.926468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 22:25:28.926468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 22:25:28.926468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 22:25:28.926468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 22:25:28.926468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise self.value Aug 29 22:25:28.926468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 22:25:28.926468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 22:25:28.926468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 22:25:28.926468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application session.commit() Aug 29 22:25:28.926468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 22:25:28.927633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 22:25:28.927633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 22:25:28.927633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 22:25:28.927633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 22:25:28.927633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self.session.flush() Aug 29 22:25:28.927633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 22:25:28.927633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 22:25:28.927633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 22:25:28.927633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 22:25:28.927633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 22:25:28.927633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application compat.raise_( Aug 29 22:25:28.927633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 22:25:28.927633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception Aug 29 22:25:28.927633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 22:25:28.927633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 22:25:28.927633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 22:25:28.927633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 22:25:28.930513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 22:25:28.930513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 22:25:28.930513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 22:25:28.930513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 22:25:28.930513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 22:25:28.930513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 22:25:28.930513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 22:25:28.930513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 22:25:28.930513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 22:25:28.930513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 22:25:28.930513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 22:25:28.930513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 22:25:28.930513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 22:25:28.930513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 22:25:28.930513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 22:25:28.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 22:25:28.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 22:25:28.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception Aug 29 22:25:28.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 22:25:28.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 22:25:28.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 22:25:28.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 22:25:28.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 22:25:28.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 22:25:28.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 22:25:28.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application conn.query(q) Aug 29 22:25:28.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 22:25:28.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 22:25:28.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 22:25:28.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application result.read() Aug 29 22:25:28.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 22:25:28.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 22:25:28.934609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 22:25:28.934609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 22:25:28.934609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 22:25:28.934609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 22:25:28.934609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 22:25:28.934609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 22:25:28.934609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 22:25:28.934609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 22:25:28.934609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [parameters: {'id': '929ed6dc6b4842f8806e23a456d2c60c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 22:25:28.934609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 22:25:28.934609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:28.934609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:28.934609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:28.934609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:28.934609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 29 22:25:28.936603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 29 22:25:28.936603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 22:25:28.936603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 22:25:28.936603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 29 22:25:28.936603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:28.936603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:28.936603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:28.936603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:28.936603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:28.936603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:28.936603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:28.936603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:28.936603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:28.936603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:28.936603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:28.936603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:28.936603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:28.938890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:28.938890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 29 22:25:28.938890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 29 22:25:28.938890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:28.938890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:28.938890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 29 22:25:28.938890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 29 22:25:28.938890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 929ed6dc6b4842f8806e23a456d2c60c. Aug 29 22:25:28.938890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:28.938890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 479/936] 10.222.0.59 () {60 vars in 1140 bytes} [Mon Aug 29 22:25:28 2022] POST /identity/v3/projects => generated 262 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:28.940318 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5396107-825d-443c-b5bc-16f17161658d None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:29.025999 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8408d-1610-4c3b-9cb4-5fffa7b44548 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:29.026300 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8408d-1610-4c3b-9cb4-5fffa7b44548 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:29.026460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8408d-1610-4c3b-9cb4-5fffa7b44548 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:29.031708 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 480/937] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:25:29.033375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8408d-1610-4c3b-9cb4-5fffa7b44548 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:29.054645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5396107-825d-443c-b5bc-16f17161658d None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:29.066863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:29.093949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a2eac-5cd6-403d-99a5-ed511f6d910b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:29.094871 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a2eac-5cd6-403d-99a5-ed511f6d910b None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:29.095115 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a2eac-5cd6-403d-99a5-ed511f6d910b None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:29.095373 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a2eac-5cd6-403d-99a5-ed511f6d910b None admin] PATH_INFO: `/v3/projects/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:29.102833 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a2eac-5cd6-403d-99a5-ed511f6d910b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:29.104444 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a2eac-5cd6-403d-99a5-ed511f6d910b None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:29.105678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:29.117488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-a86a2eac-5cd6-403d-99a5-ed511f6d910b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:29.117488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:29.117488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:29.117488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:29.117488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:29.117488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:29.117488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:29.117488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:29.117488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:29.117488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:29.117488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:29.117488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:29.117488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:25:29.117488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:25:29.117488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:25:29.118429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:25:29.118429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:29.118429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:29.118429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:29.118429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:29.118429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:29.118429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:29.118429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:29.118429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:29.118429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:29.118429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:29.118429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:29.118429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:29.118429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:29.118429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:29.118429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:29.118429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:29.124605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:25:29.124605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:25:29.124605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:25:29.124605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:25:29.124605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:25:29.124605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:25:29.124605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:29.124605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:29.124605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 458/938] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:29.124605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a7770-4344-4b6f-a3c2-5ae6e7522c9c None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:29.124605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a7770-4344-4b6f-a3c2-5ae6e7522c9c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:29.125247 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a7770-4344-4b6f-a3c2-5ae6e7522c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:29.139882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-2f8a7770-4344-4b6f-a3c2-5ae6e7522c9c None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:29.141184 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-2f8a7770-4344-4b6f-a3c2-5ae6e7522c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:29.155543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8408d-1610-4c3b-9cb4-5fffa7b44548 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:29.190978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf60b8-9f78-4f1b-8f3e-42b67a4944f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:29.192070 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf60b8-9f78-4f1b-8f3e-42b67a4944f4 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:29.192607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf60b8-9f78-4f1b-8f3e-42b67a4944f4 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:29.192875 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf60b8-9f78-4f1b-8f3e-42b67a4944f4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:29.206150 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bf60b8-9f78-4f1b-8f3e-42b67a4944f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:25:29.212290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-15bf60b8-9f78-4f1b-8f3e-42b67a4944f4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:25:29.212290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:29.212290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:29.212290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:29.212290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:29.212290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:29.212290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:29.212290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:29.212290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:29.212290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:29.212290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:29.212290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:29.212290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:25:29.212290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:25:29.212290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:25:29.212290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:25:29.213296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:25:29.213296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:25:29.213296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:25:29.213296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:25:29.213296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:29.213296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:29.213296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:29.213296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:29.213296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:29.213296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:29.213296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:29.213296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:29.213296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:29.213296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:29.213296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:29.213296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:29.214166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:29.214166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:29.214166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:29.214166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:29.214166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:25:29.214166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:25:29.214166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:25:29.214166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:25:29.214166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:25:29.214166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:25:29.214166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:25:29.214166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:29.214804 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 459/939] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:29.214804 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a7770-4344-4b6f-a3c2-5ae6e7522c9c None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:29.220838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 481/940] 10.222.0.59 () {56 vars in 1120 bytes} [Mon Aug 29 22:25:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:29.229434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf60b8-9f78-4f1b-8f3e-42b67a4944f4 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:29.346398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf60b8-9f78-4f1b-8f3e-42b67a4944f4 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:29.348494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:29.353948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a7770-4344-4b6f-a3c2-5ae6e7522c9c None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:29.377010 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:29.391860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf8618-d052-4fa5-bfa5-50d3c2583ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:29.393044 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf8618-d052-4fa5-bfa5-50d3c2583ae1 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:29.393468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf8618-d052-4fa5-bfa5-50d3c2583ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:29.394068 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf8618-d052-4fa5-bfa5-50d3c2583ae1 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:29.394977 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf8618-d052-4fa5-bfa5-50d3c2583ae1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:29.395497 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf8618-d052-4fa5-bfa5-50d3c2583ae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:29.398132 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf8618-d052-4fa5-bfa5-50d3c2583ae1 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:29.399558 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5513495-6402-4fad-8376-056f875fc4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:29.400367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5513495-6402-4fad-8376-056f875fc4ba None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:29.400655 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5513495-6402-4fad-8376-056f875fc4ba None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:29.400928 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5513495-6402-4fad-8376-056f875fc4ba None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:29.401602 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5513495-6402-4fad-8376-056f875fc4ba None admin] RBAC: Adding query filter params (name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:29.402254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5513495-6402-4fad-8376-056f875fc4ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:29.403756 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5513495-6402-4fad-8376-056f875fc4ba None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:29.416070 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 482/941] 10.222.0.59 () {56 vars in 1115 bytes} [Mon Aug 29 22:25:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:29.417967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf8618-d052-4fa5-bfa5-50d3c2583ae1 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:29.428226 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 460/942] 10.222.0.59 () {56 vars in 1112 bytes} [Mon Aug 29 22:25:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:29.486093 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5513495-6402-4fad-8376-056f875fc4ba None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:29.545506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf8618-d052-4fa5-bfa5-50d3c2583ae1 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:29.593640 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb231a5-2c98-4f94-82f7-5a96bec47460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:29.594299 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb231a5-2c98-4f94-82f7-5a96bec47460 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:29.594585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb231a5-2c98-4f94-82f7-5a96bec47460 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:29.594860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb231a5-2c98-4f94-82f7-5a96bec47460 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:29.608863 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb231a5-2c98-4f94-82f7-5a96bec47460 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:29.610254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb231a5-2c98-4f94-82f7-5a96bec47460 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:29.627361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-0cb231a5-2c98-4f94-82f7-5a96bec47460 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:29.627361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:29.627361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:29.627361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:29.627361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:29.627361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:29.627361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:29.627361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:29.627361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:29.627361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:29.627361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:29.627361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:29.627361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:29.627361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:29.627361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:29.627361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:29.628367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:29.628367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:29.628367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:29.628367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:29.628367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:29.628367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:29.628367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:29.628367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:29.628367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:29.628367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:29.628367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:29.628367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:29.628367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:29.628367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:29.628367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:29.628367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:29.628367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:29.629252 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:29.629252 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:29.629252 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:29.629252 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 483/943] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:29.631754 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5513495-6402-4fad-8376-056f875fc4ba None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:29.638187 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb231a5-2c98-4f94-82f7-5a96bec47460 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:29.685973 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14412c3c-923d-44c6-a029-375390d12f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:29.686989 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14412c3c-923d-44c6-a029-375390d12f93 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:29.687274 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14412c3c-923d-44c6-a029-375390d12f93 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:29.687550 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14412c3c-923d-44c6-a029-375390d12f93 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:29.688215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14412c3c-923d-44c6-a029-375390d12f93 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:29.689942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14412c3c-923d-44c6-a029-375390d12f93 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:29.696342 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:29.700705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-14412c3c-923d-44c6-a029-375390d12f93 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:25:29.700705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:29.700705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:29.700705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:29.700705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:29.700705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:29.700705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:29.700705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:29.700705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:25:29.700705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:29.700705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:29.700705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:29.700705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:29.700705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:29.700705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:29.700705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:29.700705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:29.701757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:29.701757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:29.701757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:29.701757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:29.701757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:29.701757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:29.701757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:29.701757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:29.701757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:29.701757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:29.701757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:29.701757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:29.701757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:29.701757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:29.701757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:29.701757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:29.702586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:29.702586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:29.702586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:29.702586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:29.702586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:29.702586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:29.702586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:29.702586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:29.702586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:29.702586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:29.702586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:29.702586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:29.702586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:25:29.702586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:29.702586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 461/944] 10.222.0.59 () {56 vars in 1126 bytes} [Mon Aug 29 22:25:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:29.710281 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14412c3c-923d-44c6-a029-375390d12f93 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:29.748751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:29.781313 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb231a5-2c98-4f94-82f7-5a96bec47460 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:29.823734 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab2f8f-8a6f-447e-8df4-37b2c479cb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:29.824403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab2f8f-8a6f-447e-8df4-37b2c479cb79 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:29.824689 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab2f8f-8a6f-447e-8df4-37b2c479cb79 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:29.824928 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab2f8f-8a6f-447e-8df4-37b2c479cb79 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:29.825567 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab2f8f-8a6f-447e-8df4-37b2c479cb79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:29.825861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab2f8f-8a6f-447e-8df4-37b2c479cb79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:29.827234 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab2f8f-8a6f-447e-8df4-37b2c479cb79 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:29.840078 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 484/945] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:29.845276 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab2f8f-8a6f-447e-8df4-37b2c479cb79 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:29.850470 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14412c3c-923d-44c6-a029-375390d12f93 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:29.899892 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762cf16-a7ed-46e9-9169-b94d7d780930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:29.900869 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762cf16-a7ed-46e9-9169-b94d7d780930 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:29.901315 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762cf16-a7ed-46e9-9169-b94d7d780930 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:29.902228 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762cf16-a7ed-46e9-9169-b94d7d780930 None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:29.903058 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3762cf16-a7ed-46e9-9169-b94d7d780930 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:29.903926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3762cf16-a7ed-46e9-9169-b94d7d780930 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:29.905815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3762cf16-a7ed-46e9-9169-b94d7d780930 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:29.915133 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 462/946] 10.222.0.59 () {56 vars in 1122 bytes} [Mon Aug 29 22:25:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:29.921869 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762cf16-a7ed-46e9-9169-b94d7d780930 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:29.994733 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:30.000727 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab2f8f-8a6f-447e-8df4-37b2c479cb79 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:30.033951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:30.046808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9f3c4-9ec9-4530-97a2-a54de1f810a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:30.047693 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9f3c4-9ec9-4530-97a2-a54de1f810a3 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:30.047965 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9f3c4-9ec9-4530-97a2-a54de1f810a3 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:30.048319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9f3c4-9ec9-4530-97a2-a54de1f810a3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:30.050214 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762cf16-a7ed-46e9-9169-b94d7d780930 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:30.058414 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9f3c4-9ec9-4530-97a2-a54de1f810a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:30.060582 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9f3c4-9ec9-4530-97a2-a54de1f810a3 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:30.076546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-47e9f3c4-9ec9-4530-97a2-a54de1f810a3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:25:30.076546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:30.076546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:30.076546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:30.076546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:30.076546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:30.076546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:30.076546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:30.076546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:30.076546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:30.076546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:30.076546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:30.076546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:25:30.076546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:25:30.076546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:25:30.076546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:25:30.077904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:30.077904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:30.077904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:25:30.077904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:30.077904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:25:30.077904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:30.077904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:30.077904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:30.077904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:30.077904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:30.077904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:30.077904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:30.077904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:30.077904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:30.077904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:30.077904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:30.077904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:30.077904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:30.079591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:30.079591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:30.079591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:25:30.079591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:25:30.079591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:25:30.079591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:25:30.079591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:25:30.079591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:25:30.079591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:25:30.079591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:30.084769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 485/947] 10.222.0.59 () {56 vars in 1117 bytes} [Mon Aug 29 22:25:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:30.091070 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9f3c4-9ec9-4530-97a2-a54de1f810a3 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:30.113348 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ead913-9494-4874-96df-3884472594a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:30.114002 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ead913-9494-4874-96df-3884472594a1 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:30.114282 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ead913-9494-4874-96df-3884472594a1 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:30.114552 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ead913-9494-4874-96df-3884472594a1 None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:30.115255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ead913-9494-4874-96df-3884472594a1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:30.115798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ead913-9494-4874-96df-3884472594a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:30.117526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ead913-9494-4874-96df-3884472594a1 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:30.128282 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 463/948] 10.222.0.59 () {56 vars in 1122 bytes} [Mon Aug 29 22:25:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:30.129271 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ead913-9494-4874-96df-3884472594a1 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:30.239137 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9f3c4-9ec9-4530-97a2-a54de1f810a3 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:30.259831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ead913-9494-4874-96df-3884472594a1 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:30.279074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf3959-0dab-42a2-a2da-ef2febef4810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:30.279632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf3959-0dab-42a2-a2da-ef2febef4810 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:30.279915 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf3959-0dab-42a2-a2da-ef2febef4810 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:30.280210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf3959-0dab-42a2-a2da-ef2febef4810 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:30.281272 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf3959-0dab-42a2-a2da-ef2febef4810 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:30.281272 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf3959-0dab-42a2-a2da-ef2febef4810 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:30.282653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf3959-0dab-42a2-a2da-ef2febef4810 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:30.307493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 486/949] 10.222.0.59 () {56 vars in 1149 bytes} [Mon Aug 29 22:25:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:30.309028 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf3959-0dab-42a2-a2da-ef2febef4810 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:30.313879 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974549a-ea70-4c85-92a4-47331486430d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:30.314941 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974549a-ea70-4c85-92a4-47331486430d None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:30.314941 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974549a-ea70-4c85-92a4-47331486430d None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:30.315296 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974549a-ea70-4c85-92a4-47331486430d None admin] PATH_INFO: `/v3/endpoints` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:30.316121 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974549a-ea70-4c85-92a4-47331486430d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:30.317613 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974549a-ea70-4c85-92a4-47331486430d None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:30.333235 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:30.404425 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 464/950] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:25:30 2022] POST /identity/v3/endpoints => generated 349 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:25:30.440690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf3959-0dab-42a2-a2da-ef2febef4810 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:30.492516 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb9409-4559-4d8c-b605-d0972deda5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:30.493539 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb9409-4559-4d8c-b605-d0972deda5e8 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:30.493993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb9409-4559-4d8c-b605-d0972deda5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:30.494422 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb9409-4559-4d8c-b605-d0972deda5e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:30.509072 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb9409-4559-4d8c-b605-d0972deda5e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:30.510824 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb9409-4559-4d8c-b605-d0972deda5e8 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:30.529208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-59fb9409-4559-4d8c-b605-d0972deda5e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:30.529208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:30.529208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:30.529208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:30.529208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:30.529208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:30.529208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:30.529208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:30.529208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:30.529208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:30.529208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:30.529208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:30.529208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:30.529208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:30.529208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:30.529208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:30.530434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:30.530434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:30.530434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:30.530434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:30.530434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:30.530434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:30.530434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:30.530434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:30.530434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:30.530434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:30.530434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:30.530434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:30.530434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:30.530434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:30.530434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:30.530434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:30.530434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:30.531752 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:30.531752 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:30.531752 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:30.533653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 487/951] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:30.538521 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f9bb7-95da-47ae-9002-c3a70ebe7cfb None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:30.538806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f9bb7-95da-47ae-9002-c3a70ebe7cfb None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:30.539078 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f9bb7-95da-47ae-9002-c3a70ebe7cfb None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:30.540881 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb9409-4559-4d8c-b605-d0972deda5e8 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:30.541484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 465/952] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:30.547198 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b55fa-febb-4546-85b5-aba3481f7c29 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:30.547477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b55fa-febb-4546-85b5-aba3481f7c29 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:30.547749 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b55fa-febb-4546-85b5-aba3481f7c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:30.574081 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-a63b55fa-febb-4546-85b5-aba3481f7c29 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:30.576081 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a63b55fa-febb-4546-85b5-aba3481f7c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:30.580026 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:30.666014 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:30.698271 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 466/953] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:30.724646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb9409-4559-4d8c-b605-d0972deda5e8 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:30.764208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f72ed-6191-4a6d-90a2-988639f87b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:30.765042 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f72ed-6191-4a6d-90a2-988639f87b49 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:30.765482 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f72ed-6191-4a6d-90a2-988639f87b49 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:30.765909 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f72ed-6191-4a6d-90a2-988639f87b49 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:30.766680 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f72ed-6191-4a6d-90a2-988639f87b49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:30.767136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f72ed-6191-4a6d-90a2-988639f87b49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:30.768669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f72ed-6191-4a6d-90a2-988639f87b49 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:30.781981 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 488/954] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:30.789111 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b55fa-febb-4546-85b5-aba3481f7c29 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:30.964875 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b55fa-febb-4546-85b5-aba3481f7c29 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:31.025887 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a505584-66d2-4263-9b96-7c370a8b76a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:31.026851 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a505584-66d2-4263-9b96-7c370a8b76a4 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:31.027306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a505584-66d2-4263-9b96-7c370a8b76a4 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:31.027735 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a505584-66d2-4263-9b96-7c370a8b76a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:31.039880 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a505584-66d2-4263-9b96-7c370a8b76a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:31.042556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a505584-66d2-4263-9b96-7c370a8b76a4 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:31.045351 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:31.063883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-6a505584-66d2-4263-9b96-7c370a8b76a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:31.063883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:31.063883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:31.063883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:31.063883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:31.063883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:31.063883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:31.063883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:31.063883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:31.063883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:31.063883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:31.063883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:31.063883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:25:31.063883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:25:31.063883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:25:31.072865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:25:31.072865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:31.072865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:31.072865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:31.072865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:31.072865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:31.072865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:31.072865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:31.072865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:31.072865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:31.072865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:31.072865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:31.072865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:31.072865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:31.072865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:31.072865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:31.072865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:31.073759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:25:31.073759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:25:31.073759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:25:31.073759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:25:31.073759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:25:31.073759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:25:31.073759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:31.073759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:31.073759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 467/955] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:31.074345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f72ed-6191-4a6d-90a2-988639f87b49 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:31.185077 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f72ed-6191-4a6d-90a2-988639f87b49 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:31.210495 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:31.217503 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a505584-66d2-4263-9b96-7c370a8b76a4 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:31.222798 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eca7c2-a0bf-4806-8c2e-688b34b78571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:31.223543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eca7c2-a0bf-4806-8c2e-688b34b78571 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:31.223815 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eca7c2-a0bf-4806-8c2e-688b34b78571 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:31.224081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eca7c2-a0bf-4806-8c2e-688b34b78571 None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:31.225350 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eca7c2-a0bf-4806-8c2e-688b34b78571 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:31.225648 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eca7c2-a0bf-4806-8c2e-688b34b78571 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:31.227139 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eca7c2-a0bf-4806-8c2e-688b34b78571 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:31.239653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 489/956] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:25:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:31.247491 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eca7c2-a0bf-4806-8c2e-688b34b78571 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:31.359878 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eca7c2-a0bf-4806-8c2e-688b34b78571 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:31.396959 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a505584-66d2-4263-9b96-7c370a8b76a4 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:31.398988 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f469fa-b878-4605-bad4-de0102df9aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:31.399974 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f469fa-b878-4605-bad4-de0102df9aa9 None admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:31.400519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f469fa-b878-4605-bad4-de0102df9aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:31.401004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f469fa-b878-4605-bad4-de0102df9aa9 None admin] PATH_INFO: `/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/980157efae8f43b59dc063fda04c0348/roles/04230301829e46deb6a9a0f33b868e35` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:31.403534 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f469fa-b878-4605-bad4-de0102df9aa9 None admin] RBAC: Authorizing `identity:create_grant(project_id=7799820fc7c64fd6a99a0a9ccf1338e3, user_id=980157efae8f43b59dc063fda04c0348, role_id=04230301829e46deb6a9a0f33b868e35)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:31.405165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f469fa-b878-4605-bad4-de0102df9aa9 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:31.447124 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b59fc-eabd-44e2-8bd3-1aba9cde6d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:31.448105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b59fc-eabd-44e2-8bd3-1aba9cde6d74 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:31.448105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b59fc-eabd-44e2-8bd3-1aba9cde6d74 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:31.448669 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b59fc-eabd-44e2-8bd3-1aba9cde6d74 None admin] PATH_INFO: `/v3/services/compute` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:31.448937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b59fc-eabd-44e2-8bd3-1aba9cde6d74 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:31.450259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b59fc-eabd-44e2-8bd3-1aba9cde6d74 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:31.460426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-067b59fc-eabd-44e2-8bd3-1aba9cde6d74 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 22:25:31.460426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:31.460426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:31.460426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:31.460426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:31.460426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:31.460426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:31.460426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:31.460426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 22:25:31.460426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:31.460426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:31.460426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:31.460426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:31.460426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:31.460426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:31.460426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:31.460426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:31.462715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:31.462715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:31.462715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:31.462715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:31.462715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:31.462715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:31.462715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:31.462715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:31.462715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:31.462715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:31.462715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:31.462715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:31.462715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:31.462715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:31.462715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:31.462715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:31.463636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:31.463636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:31.463636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:31.463636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:31.463636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:31.463636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:31.463636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:31.463636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:31.463636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:31.463636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:31.463636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:31.463636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:31.463636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 22:25:31.463636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:31.463636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 468/957] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:31.468833 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b59fc-eabd-44e2-8bd3-1aba9cde6d74 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:31.505066 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 490/958] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:25:31 2022] PUT /identity/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/980157efae8f43b59dc063fda04c0348/roles/04230301829e46deb6a9a0f33b868e35 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:25:31.567605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:31.617581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b59fc-eabd-44e2-8bd3-1aba9cde6d74 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:31.645463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39decb-8f65-4a0a-b02e-ed3b93459487 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:31.645739 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39decb-8f65-4a0a-b02e-ed3b93459487 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:31.647321 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39decb-8f65-4a0a-b02e-ed3b93459487 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:31.652366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 491/959] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:25:31.654298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b228a59-34de-4e16-9ce4-da3492ffbd48 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:31.654589 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b228a59-34de-4e16-9ce4-da3492ffbd48 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:31.654878 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b228a59-34de-4e16-9ce4-da3492ffbd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:31.660445 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ea704-523a-4fcc-8b12-7ac2a86efd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:31.661165 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ea704-523a-4fcc-8b12-7ac2a86efd23 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:31.661444 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ea704-523a-4fcc-8b12-7ac2a86efd23 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:31.662646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ea704-523a-4fcc-8b12-7ac2a86efd23 None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:31.663505 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ea704-523a-4fcc-8b12-7ac2a86efd23 None admin] RBAC: Adding query filter params (name=compute) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:31.663854 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ea704-523a-4fcc-8b12-7ac2a86efd23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:31.665258 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ea704-523a-4fcc-8b12-7ac2a86efd23 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:31.672702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 469/960] 10.222.0.59 () {56 vars in 1112 bytes} [Mon Aug 29 22:25:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:31.678470 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ea704-523a-4fcc-8b12-7ac2a86efd23 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:31.681670 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-4b228a59-34de-4e16-9ce4-da3492ffbd48 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:31.683212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-4b228a59-34de-4e16-9ce4-da3492ffbd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:31.738926 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:31.782075 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 492/961] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:31.809466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ea704-523a-4fcc-8b12-7ac2a86efd23 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:31.839854 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:31.849696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a75b4-d60c-4ba5-9616-424b5fbda077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:31.850419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a75b4-d60c-4ba5-9616-424b5fbda077 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:31.850699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a75b4-d60c-4ba5-9616-424b5fbda077 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:31.850973 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a75b4-d60c-4ba5-9616-424b5fbda077 None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:31.851634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a75b4-d60c-4ba5-9616-424b5fbda077 None admin] RBAC: Adding query filter params (type=compute) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:31.851935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a75b4-d60c-4ba5-9616-424b5fbda077 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:31.853625 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a75b4-d60c-4ba5-9616-424b5fbda077 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:31.859415 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 470/962] 10.222.0.59 () {56 vars in 1112 bytes} [Mon Aug 29 22:25:31 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:32.082447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b228a59-34de-4e16-9ce4-da3492ffbd48 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:32.201162 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b228a59-34de-4e16-9ce4-da3492ffbd48 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:32.242757 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd5597-a2b3-4925-a5c0-245f2b4a5ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:32.243760 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd5597-a2b3-4925-a5c0-245f2b4a5ce7 None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:32.244282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd5597-a2b3-4925-a5c0-245f2b4a5ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:32.244757 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd5597-a2b3-4925-a5c0-245f2b4a5ce7 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:32.245635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd5597-a2b3-4925-a5c0-245f2b4a5ce7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:32.247772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd5597-a2b3-4925-a5c0-245f2b4a5ce7 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:32.307687 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.sql.core [None req-9bfd5597-a2b3-4925-a5c0-245f2b4a5ce7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 22:25:32.307687 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 22:25:32.307687 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [parameters: {'user_id': '35ea6cb893db486baf3c847db93adef5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 22:25:32.307687 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=82165) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 22:25:32.316244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-9bfd5597-a2b3-4925-a5c0-245f2b4a5ce7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 22:25:32.316244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:32.316244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 22:25:32.316244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 22:25:32.316244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 22:25:32.316244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 22:25:32.316244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 22:25:32.316244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 22:25:32.316244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 22:25:32.316244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application conn.query(q) Aug 29 22:25:32.316244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 22:25:32.316244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 22:25:32.316244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 22:25:32.316244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application result.read() Aug 29 22:25:32.316244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 22:25:32.317357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 22:25:32.317357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 22:25:32.317357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 22:25:32.317357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 22:25:32.317357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 22:25:32.317357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 22:25:32.317357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 22:25:32.317357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 22:25:32.317357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:32.317357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 22:25:32.317357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:32.317357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:32.317357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 22:25:32.317357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 22:25:32.317357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 29 22:25:32.317357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 29 22:25:32.317357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 22:25:32.317357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application next(self.gen) Aug 29 22:25:32.318585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 22:25:32.318585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application yield resource Aug 29 22:25:32.318585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 22:25:32.318585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application next(self.gen) Aug 29 22:25:32.318585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 22:25:32.318585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 22:25:32.318585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 22:25:32.318585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 22:25:32.318585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 22:25:32.318585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise self.value Aug 29 22:25:32.318585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 22:25:32.318585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 22:25:32.318585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 22:25:32.318585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application session.commit() Aug 29 22:25:32.318585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 22:25:32.318585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 22:25:32.318585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 22:25:32.320353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 22:25:32.320353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 22:25:32.320353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self.session.flush() Aug 29 22:25:32.320353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 22:25:32.320353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 22:25:32.320353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 22:25:32.320353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 22:25:32.320353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 22:25:32.320353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application compat.raise_( Aug 29 22:25:32.320353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 22:25:32.320353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception Aug 29 22:25:32.320353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 22:25:32.320353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 22:25:32.320353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 22:25:32.320353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 22:25:32.320353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 22:25:32.320353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 22:25:32.321563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 22:25:32.321563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 22:25:32.321563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 29 22:25:32.321563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 29 22:25:32.321563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 22:25:32.321563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 22:25:32.321563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 22:25:32.321563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 22:25:32.321563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 22:25:32.321563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 22:25:32.321563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 22:25:32.321563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 22:25:32.321563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 22:25:32.321563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 22:25:32.321563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 22:25:32.321563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception Aug 29 22:25:32.322585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 22:25:32.322585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 22:25:32.322585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 22:25:32.322585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 22:25:32.322585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 22:25:32.322585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 22:25:32.322585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 22:25:32.322585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application conn.query(q) Aug 29 22:25:32.322585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 22:25:32.322585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 22:25:32.322585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 22:25:32.322585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application result.read() Aug 29 22:25:32.322585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 22:25:32.322585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 22:25:32.322585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 22:25:32.322585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 22:25:32.323416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 22:25:32.323416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 22:25:32.323416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 22:25:32.323416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 22:25:32.323416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 22:25:32.323416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 22:25:32.323416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [parameters: {'user_id': '35ea6cb893db486baf3c847db93adef5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 22:25:32.323416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 22:25:32.323416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:32.323416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:32.323416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:32.323416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:32.323416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:32.323416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:32.323416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:32.327789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:32.327789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:32.327789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:32.327789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:32.327789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:32.327789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:32.327789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:32.327789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 29 22:25:32.327789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 29 22:25:32.327789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:32.327789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:32.327789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:25:32.327789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:32.327789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:25:32.327789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:32.327789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 29 22:25:32.327789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 29 22:25:32.328800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 29 22:25:32.328800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 29 22:25:32.328800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 22:25:32.328800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 22:25:32.328800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 22:25:32.328800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:32.328800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 493/963] 10.222.0.59 () {60 vars in 1131 bytes} [Mon Aug 29 22:25:32 2022] POST /identity/v3/users => generated 160 bytes in 245 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:32.333136 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a75b4-d60c-4ba5-9616-424b5fbda077 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:32.451578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a75b4-d60c-4ba5-9616-424b5fbda077 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:32.491012 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c879e-5c25-4102-8ab2-6f96957bbf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:32.492251 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c879e-5c25-4102-8ab2-6f96957bbf99 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:32.492767 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c879e-5c25-4102-8ab2-6f96957bbf99 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:32.493285 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c879e-5c25-4102-8ab2-6f96957bbf99 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:32.502014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c879e-5c25-4102-8ab2-6f96957bbf99 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:32.503758 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c879e-5c25-4102-8ab2-6f96957bbf99 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:32.517048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-bf1c879e-5c25-4102-8ab2-6f96957bbf99 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 22:25:32.517048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:32.517048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:32.517048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:32.517048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:32.517048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:32.517048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:32.517048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:32.517048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:32.517048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:32.517048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:32.517048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:32.517048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:25:32.517048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:25:32.517048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:25:32.517048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:25:32.519065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:32.519065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:32.519065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:25:32.519065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:32.519065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:25:32.519065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:32.519065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:32.519065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:32.519065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:32.519065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:32.519065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:32.519065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:32.519065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:32.519065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:32.519065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:32.519065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:32.519065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:32.519065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:32.520266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:32.520266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:32.520266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:25:32.520266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:25:32.520266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:25:32.520266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:25:32.520266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:25:32.520266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:25:32.520266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 22:25:32.520266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:32.522783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 471/964] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:25:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:32.528726 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd5597-a2b3-4925-a5c0-245f2b4a5ce7 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:32.572088 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:32.654753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd5597-a2b3-4925-a5c0-245f2b4a5ce7 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:32.709165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943370c-befb-472e-b17f-3e9a960b47f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:32.710631 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943370c-befb-472e-b17f-3e9a960b47f9 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:32.711062 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943370c-befb-472e-b17f-3e9a960b47f9 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:32.711484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943370c-befb-472e-b17f-3e9a960b47f9 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:32.712405 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943370c-befb-472e-b17f-3e9a960b47f9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:32.712855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943370c-befb-472e-b17f-3e9a960b47f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:32.714698 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943370c-befb-472e-b17f-3e9a960b47f9 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:32.758712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 494/965] 10.222.0.59 () {56 vars in 1101 bytes} [Mon Aug 29 22:25:32 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:32.810881 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6497a4-5f40-4d41-8a1a-cb9e19b5ed39 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:32.810881 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6497a4-5f40-4d41-8a1a-cb9e19b5ed39 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:32.810881 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6497a4-5f40-4d41-8a1a-cb9e19b5ed39 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:32.811559 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 472/966] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:32.817565 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c13c9-acf8-4ef2-8f25-c05154a04d8b None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:32.817879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c13c9-acf8-4ef2-8f25-c05154a04d8b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:32.818156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c13c9-acf8-4ef2-8f25-c05154a04d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:32.840111 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-5c3c13c9-acf8-4ef2-8f25-c05154a04d8b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:32.841885 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-5c3c13c9-acf8-4ef2-8f25-c05154a04d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:32.944826 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 495/967] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:33.243202 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6497a4-5f40-4d41-8a1a-cb9e19b5ed39 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:33.323553 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:33.383728 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6497a4-5f40-4d41-8a1a-cb9e19b5ed39 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:33.423491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a8178-3ce9-4f10-acbd-6f104096fd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:33.424178 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a8178-3ce9-4f10-acbd-6f104096fd89 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:33.424426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a8178-3ce9-4f10-acbd-6f104096fd89 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:33.424691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a8178-3ce9-4f10-acbd-6f104096fd89 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:33.425271 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a8178-3ce9-4f10-acbd-6f104096fd89 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:33.426685 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a8178-3ce9-4f10-acbd-6f104096fd89 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:33.436945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-387a8178-3ce9-4f10-acbd-6f104096fd89 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:25:33.436945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:33.436945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:33.436945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:33.436945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:33.436945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:33.436945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:33.436945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:33.436945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:25:33.436945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:33.436945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:33.436945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:33.436945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:33.436945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:33.436945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:33.436945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:33.436945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:33.438888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:33.438888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:33.438888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:33.438888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:33.438888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:33.438888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:33.438888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:33.438888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:33.438888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:33.438888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:33.438888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:33.438888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:33.438888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:33.438888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:33.438888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:33.438888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:33.440970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:33.440970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:33.440970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:33.440970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:33.440970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:33.440970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:33.440970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:33.440970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:33.440970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:33.440970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:33.440970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:33.440970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:33.440970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:25:33.440970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:33.442530 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 473/968] 10.222.0.59 () {56 vars in 1126 bytes} [Mon Aug 29 22:25:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:33.445811 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c13c9-acf8-4ef2-8f25-c05154a04d8b None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:33.479454 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:33.579141 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c13c9-acf8-4ef2-8f25-c05154a04d8b None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:33.608150 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b232de-01c2-4980-99c1-d68df084f56b None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:33.608704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b232de-01c2-4980-99c1-d68df084f56b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:33.608704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b232de-01c2-4980-99c1-d68df084f56b None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:33.610149 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 474/969] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:25:33.617879 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fa4c9-26b8-463f-81c4-f522917a247c None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:33.617879 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fa4c9-26b8-463f-81c4-f522917a247c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:33.617879 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fa4c9-26b8-463f-81c4-f522917a247c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:33.635487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeeb5ff-45c2-4425-8457-a68d0b073f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:33.636093 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeeb5ff-45c2-4425-8457-a68d0b073f1e None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:33.636395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeeb5ff-45c2-4425-8457-a68d0b073f1e None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:33.636662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeeb5ff-45c2-4425-8457-a68d0b073f1e None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:33.637101 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-4f8fa4c9-26b8-463f-81c4-f522917a247c None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:33.637318 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeeb5ff-45c2-4425-8457-a68d0b073f1e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:33.637662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeeb5ff-45c2-4425-8457-a68d0b073f1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:33.638795 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-4f8fa4c9-26b8-463f-81c4-f522917a247c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:33.639046 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeeb5ff-45c2-4425-8457-a68d0b073f1e None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:33.644863 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 496/970] 10.222.0.59 () {56 vars in 1122 bytes} [Mon Aug 29 22:25:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:33.651605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeeb5ff-45c2-4425-8457-a68d0b073f1e None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:33.744177 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 475/971] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:33.769801 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:33.787384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeeb5ff-45c2-4425-8457-a68d0b073f1e None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:33.838156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb74a9d-3b7a-4919-bf29-a1ebf42cd10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:33.838687 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb74a9d-3b7a-4919-bf29-a1ebf42cd10b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:33.838975 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb74a9d-3b7a-4919-bf29-a1ebf42cd10b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:33.839233 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb74a9d-3b7a-4919-bf29-a1ebf42cd10b None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:33.839845 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb74a9d-3b7a-4919-bf29-a1ebf42cd10b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:33.840127 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb74a9d-3b7a-4919-bf29-a1ebf42cd10b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:33.841589 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb74a9d-3b7a-4919-bf29-a1ebf42cd10b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:33.847282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 497/972] 10.222.0.59 () {56 vars in 1122 bytes} [Mon Aug 29 22:25:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:33.854237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8fa4c9-26b8-463f-81c4-f522917a247c None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:33.910887 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:33.992440 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfd860-dd6d-45d6-a215-680c30ed5e66 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:33.992742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfd860-dd6d-45d6-a215-680c30ed5e66 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:33.992989 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfd860-dd6d-45d6-a215-680c30ed5e66 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:33.994559 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 498/973] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:33.999297 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8fa4c9-26b8-463f-81c4-f522917a247c None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:33.999799 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6344e9b-a1ee-4593-ba5e-9eadfb13b774 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:34.000089 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6344e9b-a1ee-4593-ba5e-9eadfb13b774 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:34.000388 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6344e9b-a1ee-4593-ba5e-9eadfb13b774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:34.020179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-c6344e9b-a1ee-4593-ba5e-9eadfb13b774 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:34.021501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c6344e9b-a1ee-4593-ba5e-9eadfb13b774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:34.042461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93ef59-159b-4f01-8ab9-4a7122feb0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:34.043075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93ef59-159b-4f01-8ab9-4a7122feb0c5 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:34.043345 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93ef59-159b-4f01-8ab9-4a7122feb0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:34.043605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93ef59-159b-4f01-8ab9-4a7122feb0c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:34.044355 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93ef59-159b-4f01-8ab9-4a7122feb0c5 None admin] RBAC: Adding query filter params (interface=admin, service_id=7442c744e74a4fccacdb5da724860d02, region_id=RegionOne) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:34.044707 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93ef59-159b-4f01-8ab9-4a7122feb0c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:34.046094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93ef59-159b-4f01-8ab9-4a7122feb0c5 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:34.055364 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 476/974] 10.222.0.59 () {56 vars in 1249 bytes} [Mon Aug 29 22:25:33 2022] GET /identity/v3/endpoints?service_id=7442c744e74a4fccacdb5da724860d02&interface=admin®ion_id=RegionOne => generated 193 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:34.056196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93ef59-159b-4f01-8ab9-4a7122feb0c5 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:34.107732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 499/975] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:34.109718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6344e9b-a1ee-4593-ba5e-9eadfb13b774 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:34.189002 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93ef59-159b-4f01-8ab9-4a7122feb0c5 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:34.200007 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:34.225428 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:34.236624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c436c2-5a9c-40fb-af9e-125713da13b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:34.237340 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c436c2-5a9c-40fb-af9e-125713da13b8 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:34.237607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c436c2-5a9c-40fb-af9e-125713da13b8 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:34.237881 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c436c2-5a9c-40fb-af9e-125713da13b8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:34.251963 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6344e9b-a1ee-4593-ba5e-9eadfb13b774 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:34.252291 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c436c2-5a9c-40fb-af9e-125713da13b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:25:34.258196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-b8c436c2-5a9c-40fb-af9e-125713da13b8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:25:34.258196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:34.258196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:34.258196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:34.258196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:34.258196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:34.258196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:34.258196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:34.258196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:34.258196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:34.258196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:34.258196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:34.258196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:25:34.258196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:25:34.258196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:25:34.258196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:25:34.259189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:25:34.259189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:25:34.259189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:25:34.259189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:25:34.259189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:34.259189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:34.259189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:34.259189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:34.259189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:34.259189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:34.259189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:34.259189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:34.259189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:34.259189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:34.259189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:34.259189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:34.260609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:34.260609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:34.260609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:34.260609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:34.260609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:25:34.260609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:25:34.260609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:25:34.260609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:25:34.260609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:25:34.260609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:25:34.260609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:25:34.260609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:34.260609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 477/976] 10.222.0.59 () {56 vars in 1120 bytes} [Mon Aug 29 22:25:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:34.272267 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c436c2-5a9c-40fb-af9e-125713da13b8 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:34.298749 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadfd6f8-a83a-4ea6-93be-ffc4a598889a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:34.299354 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadfd6f8-a83a-4ea6-93be-ffc4a598889a None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:34.299626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadfd6f8-a83a-4ea6-93be-ffc4a598889a None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:34.299928 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadfd6f8-a83a-4ea6-93be-ffc4a598889a None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:34.300578 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadfd6f8-a83a-4ea6-93be-ffc4a598889a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:34.301958 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadfd6f8-a83a-4ea6-93be-ffc4a598889a None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:34.310853 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 500/977] 10.222.0.59 () {56 vars in 1087 bytes} [Mon Aug 29 22:25:34 2022] GET /identity/v3/services => generated 1497 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:25:34.390176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadfd6f8-a83a-4ea6-93be-ffc4a598889a None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:34.415124 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c436c2-5a9c-40fb-af9e-125713da13b8 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:34.471232 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6479d-fe32-4e5d-bb81-5068c731d933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:34.471232 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6479d-fe32-4e5d-bb81-5068c731d933 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:34.471232 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6479d-fe32-4e5d-bb81-5068c731d933 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:34.471585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6479d-fe32-4e5d-bb81-5068c731d933 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:34.472604 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6479d-fe32-4e5d-bb81-5068c731d933 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:34.472604 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6479d-fe32-4e5d-bb81-5068c731d933 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:34.473906 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6479d-fe32-4e5d-bb81-5068c731d933 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:34.485684 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 478/978] 10.222.0.59 () {56 vars in 1115 bytes} [Mon Aug 29 22:25:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:34.492064 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6479d-fe32-4e5d-bb81-5068c731d933 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:34.525313 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadfd6f8-a83a-4ea6-93be-ffc4a598889a None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:34.529776 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:34.572545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2d603-0649-4b38-9817-682e248955ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:34.573165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2d603-0649-4b38-9817-682e248955ae None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:34.573451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2d603-0649-4b38-9817-682e248955ae None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:34.573704 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2d603-0649-4b38-9817-682e248955ae None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:34.574353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2d603-0649-4b38-9817-682e248955ae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:34.575734 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2d603-0649-4b38-9817-682e248955ae None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:34.619070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6479d-fe32-4e5d-bb81-5068c731d933 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:34.656881 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 501/979] 10.222.0.59 () {60 vars in 1141 bytes} [Mon Aug 29 22:25:34 2022] POST /identity/v3/services => generated 242 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:25:34.673743 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b72cd-1973-48ae-9759-b02a0d1745ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:34.674643 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b72cd-1973-48ae-9759-b02a0d1745ae None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:34.674974 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b72cd-1973-48ae-9759-b02a0d1745ae None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:34.675435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b72cd-1973-48ae-9759-b02a0d1745ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:34.692578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b72cd-1973-48ae-9759-b02a0d1745ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:34.694772 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b72cd-1973-48ae-9759-b02a0d1745ae None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:34.713634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-663b72cd-1973-48ae-9759-b02a0d1745ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:34.713634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:34.713634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:34.713634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:34.713634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:34.713634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:34.713634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:34.713634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:34.713634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:34.713634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:34.713634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:34.713634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:34.713634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:34.713634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:34.713634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:34.713634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:34.714717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:34.714717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:34.714717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:34.714717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:34.714717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:34.714717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:34.714717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:34.714717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:34.714717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:34.714717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:34.714717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:34.714717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:34.714717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:34.714717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:34.714717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:34.714717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:34.714717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:34.715645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:34.715645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:34.715645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:34.715645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 479/980] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:34.721164 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2d603-0649-4b38-9817-682e248955ae None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:34.733271 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:34.855667 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2d603-0649-4b38-9817-682e248955ae None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:34.894423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935711d2-da1f-47b5-9e84-beefb1e81e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:34.895122 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935711d2-da1f-47b5-9e84-beefb1e81e4e None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:34.895403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935711d2-da1f-47b5-9e84-beefb1e81e4e None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:34.895674 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935711d2-da1f-47b5-9e84-beefb1e81e4e None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:34.897679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935711d2-da1f-47b5-9e84-beefb1e81e4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:34.897991 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935711d2-da1f-47b5-9e84-beefb1e81e4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:34.899376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935711d2-da1f-47b5-9e84-beefb1e81e4e None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:34.913433 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 502/981] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:34.918824 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b72cd-1973-48ae-9759-b02a0d1745ae None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:35.014739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:35.042376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b72cd-1973-48ae-9759-b02a0d1745ae None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:35.084773 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b55899-5a41-4591-b351-c77cbf260dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:35.085737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b55899-5a41-4591-b351-c77cbf260dcd None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:35.086017 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b55899-5a41-4591-b351-c77cbf260dcd None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:35.086284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b55899-5a41-4591-b351-c77cbf260dcd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:35.095775 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b55899-5a41-4591-b351-c77cbf260dcd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:35.097492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b55899-5a41-4591-b351-c77cbf260dcd None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:35.109715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-86b55899-5a41-4591-b351-c77cbf260dcd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:25:35.109715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:35.109715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:35.109715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:35.109715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:35.109715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:35.109715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:35.109715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:35.109715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:35.109715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:35.109715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:35.109715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:35.109715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:25:35.109715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:25:35.109715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:25:35.109715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:25:35.110765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:35.110765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:35.110765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:25:35.110765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:35.110765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:25:35.110765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:35.110765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:35.110765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:35.110765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:35.110765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:35.110765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:35.110765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:35.110765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:35.110765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:35.110765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:35.110765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:35.110765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:35.110765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:35.111737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:35.111737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:35.111737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:25:35.111737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:25:35.111737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:25:35.111737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:25:35.111737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:25:35.111737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:25:35.111737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 22:25:35.111737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:35.111737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 480/982] 10.222.0.59 () {56 vars in 1117 bytes} [Mon Aug 29 22:25:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:35.118609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935711d2-da1f-47b5-9e84-beefb1e81e4e None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:35.197486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:35.241367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935711d2-da1f-47b5-9e84-beefb1e81e4e None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:35.282059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f52c2e-7b50-4016-9185-4b800e2c8014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:35.282950 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f52c2e-7b50-4016-9185-4b800e2c8014 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:35.283367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f52c2e-7b50-4016-9185-4b800e2c8014 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:35.283771 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f52c2e-7b50-4016-9185-4b800e2c8014 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:35.284697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f52c2e-7b50-4016-9185-4b800e2c8014 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:35.285153 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f52c2e-7b50-4016-9185-4b800e2c8014 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:35.287313 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f52c2e-7b50-4016-9185-4b800e2c8014 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:35.309255 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 503/983] 10.222.0.59 () {56 vars in 1149 bytes} [Mon Aug 29 22:25:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:35.314629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b55899-5a41-4591-b351-c77cbf260dcd None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:35.437765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b55899-5a41-4591-b351-c77cbf260dcd None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:35.478594 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17482e8e-2d1e-41cb-88ce-8f53be27b77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:35.479115 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17482e8e-2d1e-41cb-88ce-8f53be27b77a None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:35.479390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17482e8e-2d1e-41cb-88ce-8f53be27b77a None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:35.479655 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17482e8e-2d1e-41cb-88ce-8f53be27b77a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:35.489054 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:35.559478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17482e8e-2d1e-41cb-88ce-8f53be27b77a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:35.560974 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17482e8e-2d1e-41cb-88ce-8f53be27b77a None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:35.682134 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 481/984] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:25:35 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:25:35.687701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f52c2e-7b50-4016-9185-4b800e2c8014 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:35.804879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f52c2e-7b50-4016-9185-4b800e2c8014 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:35.834592 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:35.855105 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605caf17-c6b1-4661-b220-53f71dca8850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:35.855763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605caf17-c6b1-4661-b220-53f71dca8850 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:35.856334 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605caf17-c6b1-4661-b220-53f71dca8850 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:35.856334 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605caf17-c6b1-4661-b220-53f71dca8850 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:35.869135 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605caf17-c6b1-4661-b220-53f71dca8850 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:35.870586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605caf17-c6b1-4661-b220-53f71dca8850 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:35.889684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-605caf17-c6b1-4661-b220-53f71dca8850 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:35.889684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:35.889684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:35.889684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:35.889684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:35.889684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:35.889684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:35.889684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:35.889684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:35.889684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:35.889684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:35.889684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:35.889684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:35.889684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:35.889684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:35.889684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:35.890992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:35.890992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:35.890992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:35.890992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:35.890992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:35.890992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:35.890992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:35.890992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:35.890992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:35.890992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:35.890992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:35.890992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:35.890992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:35.890992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:35.890992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:35.890992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:35.890992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:35.891936 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:35.891936 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:35.891936 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:35.892740 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 504/985] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:35.897845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17482e8e-2d1e-41cb-88ce-8f53be27b77a None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:35.957522 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:36.033309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17482e8e-2d1e-41cb-88ce-8f53be27b77a None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:36.080736 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811fcac-ed84-4ca6-8c9c-ae5c084b5e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:36.081364 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811fcac-ed84-4ca6-8c9c-ae5c084b5e5f None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:36.081627 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811fcac-ed84-4ca6-8c9c-ae5c084b5e5f None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:36.081902 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811fcac-ed84-4ca6-8c9c-ae5c084b5e5f None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:36.082559 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811fcac-ed84-4ca6-8c9c-ae5c084b5e5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:36.082860 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811fcac-ed84-4ca6-8c9c-ae5c084b5e5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:36.084499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811fcac-ed84-4ca6-8c9c-ae5c084b5e5f None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:36.103299 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 482/986] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:36.105812 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605caf17-c6b1-4661-b220-53f71dca8850 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:36.234925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605caf17-c6b1-4661-b220-53f71dca8850 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:36.282005 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9b50f-c863-4c33-807a-24981beaaead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:36.282643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9b50f-c863-4c33-807a-24981beaaead None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:36.282916 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9b50f-c863-4c33-807a-24981beaaead None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:36.283224 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9b50f-c863-4c33-807a-24981beaaead None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:36.291247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9b50f-c863-4c33-807a-24981beaaead None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:36.292732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9b50f-c863-4c33-807a-24981beaaead None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:36.306763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-bfa9b50f-c863-4c33-807a-24981beaaead None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:36.306763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:36.306763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:36.306763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:36.306763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:36.306763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:36.306763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:36.306763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:36.306763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:36.306763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:36.306763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:36.306763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:36.306763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:25:36.306763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:25:36.306763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:25:36.307762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:25:36.307762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:36.307762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:36.307762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:36.307762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:36.307762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:36.307762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:36.307762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:36.307762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:36.307762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:36.307762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:36.307762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:36.307762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:36.307762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:36.307762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:36.307762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:36.307762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:36.309945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:25:36.309945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:25:36.309945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:25:36.309945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:25:36.309945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:25:36.309945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:25:36.309945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:36.309945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:36.311292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 505/987] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:36.313982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811fcac-ed84-4ca6-8c9c-ae5c084b5e5f None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:36.339208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ca9da-87d6-499b-b8c3-0d3a54e9ba2f None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:36.339420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ca9da-87d6-499b-b8c3-0d3a54e9ba2f None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:36.339638 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ca9da-87d6-499b-b8c3-0d3a54e9ba2f None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:36.341375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 506/988] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:36.347063 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e69d8-8e8a-40d7-bbc7-ab510ba43172 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:36.347353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e69d8-8e8a-40d7-bbc7-ab510ba43172 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:36.347595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e69d8-8e8a-40d7-bbc7-ab510ba43172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:36.369235 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-451e69d8-8e8a-40d7-bbc7-ab510ba43172 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:36.370622 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-451e69d8-8e8a-40d7-bbc7-ab510ba43172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:36.411268 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:36.444368 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:36.448425 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811fcac-ed84-4ca6-8c9c-ae5c084b5e5f None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:36.457103 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 507/989] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:36.503829 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206bef30-4a5f-4bbd-97e3-eea7d39e5491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:36.504475 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bef30-4a5f-4bbd-97e3-eea7d39e5491 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:36.504748 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bef30-4a5f-4bbd-97e3-eea7d39e5491 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:36.505018 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206bef30-4a5f-4bbd-97e3-eea7d39e5491 None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:36.506159 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206bef30-4a5f-4bbd-97e3-eea7d39e5491 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:36.506544 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206bef30-4a5f-4bbd-97e3-eea7d39e5491 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:36.508025 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206bef30-4a5f-4bbd-97e3-eea7d39e5491 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:36.528725 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 483/990] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:25:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:36.534769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e69d8-8e8a-40d7-bbc7-ab510ba43172 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:36.663810 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e69d8-8e8a-40d7-bbc7-ab510ba43172 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:36.672323 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b016d53-141f-4cc0-8347-5dff1df57dc6 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:36.672569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b016d53-141f-4cc0-8347-5dff1df57dc6 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:36.672812 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b016d53-141f-4cc0-8347-5dff1df57dc6 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:36.674151 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 484/991] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:36.680221 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d3bfe-b12e-4248-bd25-737e186aadbd None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:36.680675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d3bfe-b12e-4248-bd25-737e186aadbd None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:36.680952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d3bfe-b12e-4248-bd25-737e186aadbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:36.699974 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-1d8d3bfe-b12e-4248-bd25-737e186aadbd None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:36.701230 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-1d8d3bfe-b12e-4248-bd25-737e186aadbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:36.717225 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e68011-2494-4c71-a29a-5cab26bd763c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:36.717778 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e68011-2494-4c71-a29a-5cab26bd763c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:36.718051 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e68011-2494-4c71-a29a-5cab26bd763c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:36.718325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e68011-2494-4c71-a29a-5cab26bd763c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:36.719187 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e68011-2494-4c71-a29a-5cab26bd763c None admin] RBAC: Adding query filter params (role.id=04230301829e46deb6a9a0f33b868e35, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=980157efae8f43b59dc063fda04c0348) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:36.719494 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e68011-2494-4c71-a29a-5cab26bd763c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:36.721209 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e68011-2494-4c71-a29a-5cab26bd763c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:36.736217 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 508/992] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:25:36 2022] GET /identity/v3/role_assignments?user.id=980157efae8f43b59dc063fda04c0348&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=04230301829e46deb6a9a0f33b868e35 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:36.736217 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e68011-2494-4c71-a29a-5cab26bd763c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:36.773167 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:36.803779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 485/993] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:36.870920 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e68011-2494-4c71-a29a-5cab26bd763c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:36.915542 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979b5f3-a570-4410-84e7-e801aebc16fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:36.916197 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979b5f3-a570-4410-84e7-e801aebc16fa None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:36.916603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979b5f3-a570-4410-84e7-e801aebc16fa None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:36.916882 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979b5f3-a570-4410-84e7-e801aebc16fa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:36.917603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979b5f3-a570-4410-84e7-e801aebc16fa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:36.919037 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979b5f3-a570-4410-84e7-e801aebc16fa None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:36.931086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-7979b5f3-a570-4410-84e7-e801aebc16fa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:25:36.931086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:36.931086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:36.931086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:36.931086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:36.931086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:36.931086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:36.931086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:36.931086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:25:36.931086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:36.931086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:36.931086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:36.931086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:36.931086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:36.931086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:36.931086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:36.931086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:36.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:36.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:36.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:36.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:36.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:36.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:36.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:36.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:36.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:36.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:36.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:36.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:36.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:36.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:36.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:36.932468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:36.933442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:36.933442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:36.933442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:36.933442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:36.933442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:36.933442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:36.933442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:36.933442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:36.933442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:36.933442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:36.933442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:36.933442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:36.933442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 22:25:36.933442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:36.933442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 509/994] 10.222.0.59 () {56 vars in 1126 bytes} [Mon Aug 29 22:25:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:36.939706 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d3bfe-b12e-4248-bd25-737e186aadbd None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:37.063422 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:37.078346 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d3bfe-b12e-4248-bd25-737e186aadbd None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:37.082106 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979b5f3-a570-4410-84e7-e801aebc16fa None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:37.127734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70a76a-a0eb-409a-8f30-0bbbd86661c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:37.128723 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70a76a-a0eb-409a-8f30-0bbbd86661c2 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:37.129179 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70a76a-a0eb-409a-8f30-0bbbd86661c2 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:37.129649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70a76a-a0eb-409a-8f30-0bbbd86661c2 None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:37.130949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70a76a-a0eb-409a-8f30-0bbbd86661c2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:37.131453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70a76a-a0eb-409a-8f30-0bbbd86661c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:37.133849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70a76a-a0eb-409a-8f30-0bbbd86661c2 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:37.146997 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 486/995] 10.222.0.59 () {56 vars in 1122 bytes} [Mon Aug 29 22:25:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:37.149069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70a76a-a0eb-409a-8f30-0bbbd86661c2 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:37.214146 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:37.223030 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979b5f3-a570-4410-84e7-e801aebc16fa None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:37.272936 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70a76a-a0eb-409a-8f30-0bbbd86661c2 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:37.273353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068322f7-d3e4-408b-9dfd-239fb7785ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:37.273634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068322f7-d3e4-408b-9dfd-239fb7785ad4 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:37.273907 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068322f7-d3e4-408b-9dfd-239fb7785ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:37.274173 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068322f7-d3e4-408b-9dfd-239fb7785ad4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:37.274757 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068322f7-d3e4-408b-9dfd-239fb7785ad4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:37.276151 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068322f7-d3e4-408b-9dfd-239fb7785ad4 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:37.289473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-068322f7-d3e4-408b-9dfd-239fb7785ad4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 22:25:37.289473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:37.289473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:37.289473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:37.289473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:37.289473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:37.289473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:37.289473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:37.289473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 22:25:37.289473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:37.289473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:37.289473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:37.289473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:37.289473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:37.289473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:37.289473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:37.289473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:37.291282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:37.291282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:37.291282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:37.291282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:37.291282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:37.291282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:37.291282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:37.291282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:37.291282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:37.291282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:37.291282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:37.291282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:37.291282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:37.291282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:37.291282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:37.291282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:37.292949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:37.292949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:37.292949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:37.292949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:37.292949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:37.292949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:37.292949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:37.292949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:37.292949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:37.292949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:37.292949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:37.292949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:37.292949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 22:25:37.292949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:37.292949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 510/996] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:37.296519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068322f7-d3e4-408b-9dfd-239fb7785ad4 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:37.320972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d78c93-84f5-428e-bf31-4441c996aa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:37.321840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d78c93-84f5-428e-bf31-4441c996aa98 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:37.322279 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d78c93-84f5-428e-bf31-4441c996aa98 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:37.322690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d78c93-84f5-428e-bf31-4441c996aa98 None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:37.323529 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d78c93-84f5-428e-bf31-4441c996aa98 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:37.324021 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d78c93-84f5-428e-bf31-4441c996aa98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:37.325721 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d78c93-84f5-428e-bf31-4441c996aa98 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:37.335452 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 487/997] 10.222.0.59 () {56 vars in 1122 bytes} [Mon Aug 29 22:25:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:37.340990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d78c93-84f5-428e-bf31-4441c996aa98 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:37.362614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:37.425905 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068322f7-d3e4-408b-9dfd-239fb7785ad4 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:37.471840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d78c93-84f5-428e-bf31-4441c996aa98 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:37.474680 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e0757c-01de-4c80-a3d6-f970d4e2a190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:37.476646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0757c-01de-4c80-a3d6-f970d4e2a190 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:37.476928 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0757c-01de-4c80-a3d6-f970d4e2a190 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:37.477201 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e0757c-01de-4c80-a3d6-f970d4e2a190 None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:37.477852 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e0757c-01de-4c80-a3d6-f970d4e2a190 None admin] RBAC: Adding query filter params (name=compute) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:37.478153 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e0757c-01de-4c80-a3d6-f970d4e2a190 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:37.479610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e0757c-01de-4c80-a3d6-f970d4e2a190 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:37.487249 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 511/998] 10.222.0.59 () {56 vars in 1112 bytes} [Mon Aug 29 22:25:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:37.494298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e0757c-01de-4c80-a3d6-f970d4e2a190 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:37.513330 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca7a4a-bc18-401d-8450-606ed4fae3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:37.513944 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca7a4a-bc18-401d-8450-606ed4fae3ff None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:37.514216 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca7a4a-bc18-401d-8450-606ed4fae3ff None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:37.514491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca7a4a-bc18-401d-8450-606ed4fae3ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:37.515142 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca7a4a-bc18-401d-8450-606ed4fae3ff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:37.517212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca7a4a-bc18-401d-8450-606ed4fae3ff None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:37.528279 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:37.597646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 488/999] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:25:37 2022] POST /identity/v3/endpoints => generated 325 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:25:37.638463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e0757c-01de-4c80-a3d6-f970d4e2a190 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:37.693808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9080a26-3f8f-4ba6-8c83-a3c4a8476088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:37.694752 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9080a26-3f8f-4ba6-8c83-a3c4a8476088 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:37.695196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9080a26-3f8f-4ba6-8c83-a3c4a8476088 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:37.695632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9080a26-3f8f-4ba6-8c83-a3c4a8476088 None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:37.696736 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9080a26-3f8f-4ba6-8c83-a3c4a8476088 None admin] RBAC: Adding query filter params (type=compute) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:37.697199 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9080a26-3f8f-4ba6-8c83-a3c4a8476088 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:37.698612 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9080a26-3f8f-4ba6-8c83-a3c4a8476088 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:37.706429 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 512/1000] 10.222.0.59 () {56 vars in 1112 bytes} [Mon Aug 29 22:25:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:37.713237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca7a4a-bc18-401d-8450-606ed4fae3ff None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:37.813316 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:37.845189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca7a4a-bc18-401d-8450-606ed4fae3ff None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:37.886671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007e2cc-6bac-4540-b713-a4086d9625c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:37.887552 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007e2cc-6bac-4540-b713-a4086d9625c4 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:37.888337 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007e2cc-6bac-4540-b713-a4086d9625c4 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:37.888619 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007e2cc-6bac-4540-b713-a4086d9625c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:37.889744 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007e2cc-6bac-4540-b713-a4086d9625c4 None admin] RBAC: Adding query filter params (interface=public, service_id=5a05b7cbe4ee414298c8712ca586ae83, region_id=RegionOne) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:37.890051 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007e2cc-6bac-4540-b713-a4086d9625c4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:37.891446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007e2cc-6bac-4540-b713-a4086d9625c4 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:37.899627 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 489/1001] 10.222.0.59 () {56 vars in 1251 bytes} [Mon Aug 29 22:25:37 2022] GET /identity/v3/endpoints?service_id=5a05b7cbe4ee414298c8712ca586ae83&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:37.905052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9080a26-3f8f-4ba6-8c83-a3c4a8476088 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:38.018496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:38.045527 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9080a26-3f8f-4ba6-8c83-a3c4a8476088 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:38.087786 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fe94c-c719-4021-8e1b-db90b13eaf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:38.088366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fe94c-c719-4021-8e1b-db90b13eaf8f None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:38.088671 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fe94c-c719-4021-8e1b-db90b13eaf8f None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:38.088937 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fe94c-c719-4021-8e1b-db90b13eaf8f None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:38.089552 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fe94c-c719-4021-8e1b-db90b13eaf8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:38.090925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fe94c-c719-4021-8e1b-db90b13eaf8f None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:38.098648 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 513/1002] 10.222.0.59 () {56 vars in 1087 bytes} [Mon Aug 29 22:25:37 2022] GET /identity/v3/services => generated 1727 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:25:38.809864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129ced6-3d3a-4ee0-b37a-b15f2d3a9f50 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:38.810489 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129ced6-3d3a-4ee0-b37a-b15f2d3a9f50 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:38.810489 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129ced6-3d3a-4ee0-b37a-b15f2d3a9f50 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:38.811955 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 490/1003] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:25:38.817432 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b853369-7f67-495d-960b-f0dde5779714 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:38.817780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b853369-7f67-495d-960b-f0dde5779714 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:38.818209 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b853369-7f67-495d-960b-f0dde5779714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:38.838514 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-6b853369-7f67-495d-960b-f0dde5779714 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:38.840426 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6b853369-7f67-495d-960b-f0dde5779714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:38.924811 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 514/1004] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:38 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:39.206842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129ced6-3d3a-4ee0-b37a-b15f2d3a9f50 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:39.342144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129ced6-3d3a-4ee0-b37a-b15f2d3a9f50 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:39.382677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c86c0b-cc4d-4b3e-9714-792b7dba19da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:39.383321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c86c0b-cc4d-4b3e-9714-792b7dba19da None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:39.383651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c86c0b-cc4d-4b3e-9714-792b7dba19da None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:39.383870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c86c0b-cc4d-4b3e-9714-792b7dba19da None admin] PATH_INFO: `/v3/services/image` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:39.384485 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c86c0b-cc4d-4b3e-9714-792b7dba19da None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:39.385863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c86c0b-cc4d-4b3e-9714-792b7dba19da None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:39.395696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-25c86c0b-cc4d-4b3e-9714-792b7dba19da None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 22:25:39.395696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:39.395696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:39.395696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:39.395696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:39.395696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:39.395696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:39.395696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:39.395696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 22:25:39.395696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:39.395696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:39.395696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:39.395696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:39.395696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:39.395696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:39.395696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:39.395696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:39.397302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:39.397302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:39.397302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:39.397302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:39.397302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:39.397302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:39.397302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:39.397302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:39.397302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:39.397302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:39.397302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:39.397302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:39.397302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:39.397302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:39.397302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:39.397302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:39.398406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:39.398406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:39.398406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:39.398406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:39.398406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:39.398406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:39.398406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:39.398406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:39.398406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:39.398406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:39.398406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:39.398406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:39.398406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 22:25:39.398406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:39.400515 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 491/1005] 10.222.0.59 () {56 vars in 1105 bytes} [Mon Aug 29 22:25:39 2022] GET /identity/v3/services/image => generated 86 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:39.403663 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b853369-7f67-495d-960b-f0dde5779714 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:39.500329 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:39.544019 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b853369-7f67-495d-960b-f0dde5779714 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:39.593111 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e21d8-d06b-48fb-bb6a-04649cfa036c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:39.593732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e21d8-d06b-48fb-bb6a-04649cfa036c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:39.594009 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e21d8-d06b-48fb-bb6a-04649cfa036c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:39.594280 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e21d8-d06b-48fb-bb6a-04649cfa036c None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:39.594924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e21d8-d06b-48fb-bb6a-04649cfa036c None admin] RBAC: Adding query filter params (name=image) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:39.595223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e21d8-d06b-48fb-bb6a-04649cfa036c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:39.596643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e21d8-d06b-48fb-bb6a-04649cfa036c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:39.608158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 515/1006] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:39.609269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c86c0b-cc4d-4b3e-9714-792b7dba19da None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:39.614010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:39.647302 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1422db-5d28-40b6-b751-58da0f788332 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:39.647638 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1422db-5d28-40b6-b751-58da0f788332 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:39.647845 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1422db-5d28-40b6-b751-58da0f788332 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:39.649436 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 516/1007] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:39.654774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096498f9-d6f3-48ba-b1dc-2e99186b9354 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:39.655067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096498f9-d6f3-48ba-b1dc-2e99186b9354 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:39.655395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096498f9-d6f3-48ba-b1dc-2e99186b9354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:39.678505 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-096498f9-d6f3-48ba-b1dc-2e99186b9354 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:39.679780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-096498f9-d6f3-48ba-b1dc-2e99186b9354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:39.756912 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c86c0b-cc4d-4b3e-9714-792b7dba19da None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:39.764114 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 517/1008] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:39 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:39.795932 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd30bc-609d-45cd-8d17-37ba6e70be08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:39.796584 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd30bc-609d-45cd-8d17-37ba6e70be08 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:39.796854 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd30bc-609d-45cd-8d17-37ba6e70be08 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:39.797119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd30bc-609d-45cd-8d17-37ba6e70be08 None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:39.797751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd30bc-609d-45cd-8d17-37ba6e70be08 None admin] RBAC: Adding query filter params (type=image) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:39.798044 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd30bc-609d-45cd-8d17-37ba6e70be08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:39.799393 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd30bc-609d-45cd-8d17-37ba6e70be08 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:39.812170 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 492/1009] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:39 2022] GET /identity/v3/services?type=image => generated 122 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:40.045592 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096498f9-d6f3-48ba-b1dc-2e99186b9354 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:40.126111 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:40.164100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096498f9-d6f3-48ba-b1dc-2e99186b9354 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:40.206570 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bdbf9-f5ee-4d1b-b2b0-4c9e6d96b4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:40.207213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bdbf9-f5ee-4d1b-b2b0-4c9e6d96b4bc None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:40.207509 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bdbf9-f5ee-4d1b-b2b0-4c9e6d96b4bc None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:40.207842 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bdbf9-f5ee-4d1b-b2b0-4c9e6d96b4bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:40.210009 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93bdbf9-f5ee-4d1b-b2b0-4c9e6d96b4bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:40.211466 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93bdbf9-f5ee-4d1b-b2b0-4c9e6d96b4bc None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:40.216204 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 518/1010] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:40.219551 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd30bc-609d-45cd-8d17-37ba6e70be08 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:40.348474 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:40.374804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e425e-5e81-40f3-8986-f0f229ad9f43 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:40.375276 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e425e-5e81-40f3-8986-f0f229ad9f43 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:40.375705 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e425e-5e81-40f3-8986-f0f229ad9f43 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:40.377296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 519/1011] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:40.383608 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050d7a6-6f43-4718-8612-e398d4ce157c None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:40.383917 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050d7a6-6f43-4718-8612-e398d4ce157c None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:40.384167 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050d7a6-6f43-4718-8612-e398d4ce157c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:40.392939 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd30bc-609d-45cd-8d17-37ba6e70be08 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:40.403107 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-b050d7a6-6f43-4718-8612-e398d4ce157c None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:40.404331 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b050d7a6-6f43-4718-8612-e398d4ce157c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:40.436222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d358592-a1ea-45e4-8609-d0df230fe8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:40.436827 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d358592-a1ea-45e4-8609-d0df230fe8f8 None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:40.437113 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d358592-a1ea-45e4-8609-d0df230fe8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:40.437439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d358592-a1ea-45e4-8609-d0df230fe8f8 None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:40.438084 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d358592-a1ea-45e4-8609-d0df230fe8f8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:40.439508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d358592-a1ea-45e4-8609-d0df230fe8f8 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:40.480986 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 520/1012] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:40 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:40.543770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 493/1013] 10.222.0.59 () {60 vars in 1141 bytes} [Mon Aug 29 22:25:40 2022] POST /identity/v3/projects => generated 311 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:25:40.763953 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050d7a6-6f43-4718-8612-e398d4ce157c None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:40.848743 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:40.886170 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050d7a6-6f43-4718-8612-e398d4ce157c None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:40.937924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b343c1b-79d5-4d08-af6c-9aed39734424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:40.938555 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b343c1b-79d5-4d08-af6c-9aed39734424 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:40.938831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b343c1b-79d5-4d08-af6c-9aed39734424 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:40.939095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b343c1b-79d5-4d08-af6c-9aed39734424 None admin] PATH_INFO: `/v3/services/compute` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:40.939706 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b343c1b-79d5-4d08-af6c-9aed39734424 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:40.941540 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b343c1b-79d5-4d08-af6c-9aed39734424 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:40.952359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-7b343c1b-79d5-4d08-af6c-9aed39734424 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 22:25:40.952359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:40.952359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:40.952359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:40.952359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:40.952359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:40.952359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:40.952359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:40.952359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 22:25:40.952359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:40.952359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:40.952359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:40.952359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:40.952359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:40.952359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:40.952359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:40.952359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:40.953457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:40.953457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:40.953457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:40.953457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:40.953457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:40.953457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:40.953457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:40.953457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:40.953457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:40.953457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:40.953457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:40.953457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:40.953457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:40.953457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:40.953457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:40.953457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:40.954333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:40.954333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:40.954333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:40.954333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:40.954333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:40.954333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:40.954333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:40.954333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:40.954333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:40.954333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:40.954333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:40.954333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:40.954333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 22:25:40.954333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:40.954333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 521/1014] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:40.960167 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d358592-a1ea-45e4-8609-d0df230fe8f8 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:41.103275 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d358592-a1ea-45e4-8609-d0df230fe8f8 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:41.136288 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:41.158259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c085571-8e77-4b7e-af57-fd563287f95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:41.158952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c085571-8e77-4b7e-af57-fd563287f95c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:41.159388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c085571-8e77-4b7e-af57-fd563287f95c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:41.159663 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c085571-8e77-4b7e-af57-fd563287f95c None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:41.160521 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c085571-8e77-4b7e-af57-fd563287f95c None admin] RBAC: Adding query filter params (name=compute) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:41.160861 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c085571-8e77-4b7e-af57-fd563287f95c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:41.162853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c085571-8e77-4b7e-af57-fd563287f95c None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:41.171458 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 494/1015] 10.222.0.59 () {56 vars in 1112 bytes} [Mon Aug 29 22:25:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:41.180127 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b343c1b-79d5-4d08-af6c-9aed39734424 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:41.301925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b343c1b-79d5-4d08-af6c-9aed39734424 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:41.345537 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:41.351708 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ed188-bb6d-485a-95ed-220ea3a3c5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:41.352361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ed188-bb6d-485a-95ed-220ea3a3c5da None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:41.352615 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ed188-bb6d-485a-95ed-220ea3a3c5da None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:41.352989 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ed188-bb6d-485a-95ed-220ea3a3c5da None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:41.360332 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ed188-bb6d-485a-95ed-220ea3a3c5da None admin] RBAC: Adding query filter params (type=compute) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:41.360332 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ed188-bb6d-485a-95ed-220ea3a3c5da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:41.360332 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ed188-bb6d-485a-95ed-220ea3a3c5da None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:41.360813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 522/1016] 10.222.0.59 () {56 vars in 1112 bytes} [Mon Aug 29 22:25:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:41.366520 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c085571-8e77-4b7e-af57-fd563287f95c None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:41.478240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c085571-8e77-4b7e-af57-fd563287f95c None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:41.518053 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf927a2-fa69-40a5-96bd-7c23176a30a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:41.518704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf927a2-fa69-40a5-96bd-7c23176a30a4 None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:41.518989 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf927a2-fa69-40a5-96bd-7c23176a30a4 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:41.519257 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf927a2-fa69-40a5-96bd-7c23176a30a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:41.519918 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf927a2-fa69-40a5-96bd-7c23176a30a4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:41.521482 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf927a2-fa69-40a5-96bd-7c23176a30a4 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:41.612855 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 495/1017] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:25:41 2022] POST /identity/v3/endpoints => generated 334 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:25:41.798016 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63044f-b28f-4a92-8380-5491abee495a None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:41.798301 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63044f-b28f-4a92-8380-5491abee495a None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:41.798576 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63044f-b28f-4a92-8380-5491abee495a None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:41.799922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 523/1018] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:41.806279 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414bea8-cfe7-4ad6-acf6-062b2bc40b3b None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:41.806650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414bea8-cfe7-4ad6-acf6-062b2bc40b3b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:41.806987 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414bea8-cfe7-4ad6-acf6-062b2bc40b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:41.826339 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-a414bea8-cfe7-4ad6-acf6-062b2bc40b3b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:41.827554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a414bea8-cfe7-4ad6-acf6-062b2bc40b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:41.867486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:41.908840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 496/1019] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:41 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:42.188437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63044f-b28f-4a92-8380-5491abee495a None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:42.311417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63044f-b28f-4a92-8380-5491abee495a None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:42.350458 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c017c-64ec-4945-a1fc-4e10270d9aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:42.351125 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c017c-64ec-4945-a1fc-4e10270d9aa6 None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:42.351404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c017c-64ec-4945-a1fc-4e10270d9aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:42.351670 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c017c-64ec-4945-a1fc-4e10270d9aa6 None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:42.352348 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2c017c-64ec-4945-a1fc-4e10270d9aa6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:42.353712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2c017c-64ec-4945-a1fc-4e10270d9aa6 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:42.446578 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 524/1020] 10.222.0.59 () {60 vars in 1141 bytes} [Mon Aug 29 22:25:42 2022] POST /identity/v3/services => generated 242 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:25:42.601103 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66787a37-18ef-467e-87b3-d5ed7ea60224 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:42.601658 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66787a37-18ef-467e-87b3-d5ed7ea60224 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:42.601658 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66787a37-18ef-467e-87b3-d5ed7ea60224 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:42.603190 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 497/1021] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:42.609071 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91740b0c-9d1b-48ee-8b78-d2584cf29a58 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:42.609391 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91740b0c-9d1b-48ee-8b78-d2584cf29a58 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:42.609643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91740b0c-9d1b-48ee-8b78-d2584cf29a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:42.629205 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-91740b0c-9d1b-48ee-8b78-d2584cf29a58 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:42.631392 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-91740b0c-9d1b-48ee-8b78-d2584cf29a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:42.686564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:42.711832 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 525/1022] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:42.990822 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66787a37-18ef-467e-87b3-d5ed7ea60224 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:43.101615 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66787a37-18ef-467e-87b3-d5ed7ea60224 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:43.140466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bdea4-625a-40a9-b872-34d60c36d3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:43.141077 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bdea4-625a-40a9-b872-34d60c36d3e5 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:43.141355 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bdea4-625a-40a9-b872-34d60c36d3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:43.141624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bdea4-625a-40a9-b872-34d60c36d3e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:43.142746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2bdea4-625a-40a9-b872-34d60c36d3e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:43.144233 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2bdea4-625a-40a9-b872-34d60c36d3e5 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:43.148205 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 498/1023] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:43.152198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91740b0c-9d1b-48ee-8b78-d2584cf29a58 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:43.261149 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91740b0c-9d1b-48ee-8b78-d2584cf29a58 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:43.298721 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b22c9c-ac2d-421c-963a-e56689746603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:43.299589 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b22c9c-ac2d-421c-963a-e56689746603 None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:43.300012 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b22c9c-ac2d-421c-963a-e56689746603 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:43.300484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b22c9c-ac2d-421c-963a-e56689746603 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:43.301385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b22c9c-ac2d-421c-963a-e56689746603 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:43.303430 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b22c9c-ac2d-421c-963a-e56689746603 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:43.422156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 526/1024] 10.222.0.59 () {60 vars in 1132 bytes} [Mon Aug 29 22:25:43 2022] POST /identity/v3/users => generated 285 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:25:43.693022 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde466ba-cd57-405c-bde9-44846b92f995 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:43.693548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde466ba-cd57-405c-bde9-44846b92f995 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:43.693626 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde466ba-cd57-405c-bde9-44846b92f995 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:43.695238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 499/1025] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:43.702190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff462e5a-6bac-4037-84a0-a303ca1e4e13 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:43.702480 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff462e5a-6bac-4037-84a0-a303ca1e4e13 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:43.702738 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff462e5a-6bac-4037-84a0-a303ca1e4e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:43.725053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-ff462e5a-6bac-4037-84a0-a303ca1e4e13 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:43.727084 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-ff462e5a-6bac-4037-84a0-a303ca1e4e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:43.762869 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:43.805178 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 527/1026] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:44.079792 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde466ba-cd57-405c-bde9-44846b92f995 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:44.122472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:44.209374 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde466ba-cd57-405c-bde9-44846b92f995 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:44.247861 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca0f52-3e53-470c-a001-f72c54bf7a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:44.248520 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca0f52-3e53-470c-a001-f72c54bf7a15 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:44.248790 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca0f52-3e53-470c-a001-f72c54bf7a15 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:44.249055 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca0f52-3e53-470c-a001-f72c54bf7a15 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:44.260940 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca0f52-3e53-470c-a001-f72c54bf7a15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:25:44.265793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-d1ca0f52-3e53-470c-a001-f72c54bf7a15 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:25:44.265793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:44.265793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:44.265793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:44.265793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:44.265793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:44.265793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:44.265793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:44.265793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:44.265793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:44.265793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:44.265793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:44.265793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:25:44.265793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:25:44.265793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:25:44.265793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:25:44.266726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:25:44.266726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:25:44.266726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:25:44.266726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:25:44.266726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:44.266726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:44.266726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:44.266726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:44.266726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:44.266726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:44.266726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:44.266726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:44.266726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:44.266726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:44.266726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:44.266726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:44.267603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:44.267603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:44.267603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:44.267603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:44.267603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:25:44.267603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:25:44.267603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:25:44.267603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:25:44.267603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:25:44.267603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:25:44.267603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:25:44.267603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:44.269284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 500/1027] 10.222.0.59 () {56 vars in 1120 bytes} [Mon Aug 29 22:25:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:44.273365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff462e5a-6bac-4037-84a0-a303ca1e4e13 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:44.397858 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff462e5a-6bac-4037-84a0-a303ca1e4e13 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:44.436788 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334af9a-ad31-4ada-891f-29646323b5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:44.437507 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334af9a-ad31-4ada-891f-29646323b5ba None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:44.437783 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334af9a-ad31-4ada-891f-29646323b5ba None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:44.438048 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334af9a-ad31-4ada-891f-29646323b5ba None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:44.438717 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334af9a-ad31-4ada-891f-29646323b5ba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:44.439017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334af9a-ad31-4ada-891f-29646323b5ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:44.440423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334af9a-ad31-4ada-891f-29646323b5ba None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:44.441702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a967b-f827-4263-931c-3b71c6e416f7 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:44.441998 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a967b-f827-4263-931c-3b71c6e416f7 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:44.442749 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a967b-f827-4263-931c-3b71c6e416f7 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:44.445107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 501/1028] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:44.454589 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 528/1029] 10.222.0.59 () {56 vars in 1115 bytes} [Mon Aug 29 22:25:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:44.454855 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec38169-734a-4bc2-ac2d-177838a78986 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:44.454855 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec38169-734a-4bc2-ac2d-177838a78986 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:44.454855 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec38169-734a-4bc2-ac2d-177838a78986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:44.456734 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334af9a-ad31-4ada-891f-29646323b5ba None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:44.471453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-0ec38169-734a-4bc2-ac2d-177838a78986 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:44.472695 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-0ec38169-734a-4bc2-ac2d-177838a78986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:44.500063 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:44.557075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 502/1030] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:44.588600 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334af9a-ad31-4ada-891f-29646323b5ba None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:44.627999 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405a9c8-f142-4903-842b-cf9a595d4988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:44.628682 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405a9c8-f142-4903-842b-cf9a595d4988 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:44.628963 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405a9c8-f142-4903-842b-cf9a595d4988 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:44.629232 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405a9c8-f142-4903-842b-cf9a595d4988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:44.644746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405a9c8-f142-4903-842b-cf9a595d4988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:44.647718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405a9c8-f142-4903-842b-cf9a595d4988 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:44.664024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-6405a9c8-f142-4903-842b-cf9a595d4988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:44.664024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:44.664024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:44.664024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:44.664024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:44.664024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:44.664024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:44.664024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:44.664024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:44.664024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:44.664024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:44.664024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:44.664024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:44.664024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:44.664024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:44.664024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:44.665067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:44.665067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:44.665067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:44.665067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:44.665067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:44.665067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:44.665067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:44.665067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:44.665067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:44.665067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:44.665067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:44.665067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:44.665067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:44.665067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:44.665067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:44.665067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:44.665067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:44.665986 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:44.665986 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:44.665986 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:44.665986 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 529/1031] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:44.671128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec38169-734a-4bc2-ac2d-177838a78986 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:44.704058 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:44.797635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec38169-734a-4bc2-ac2d-177838a78986 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:44.835843 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766071c-7a47-422b-87bd-d387b12e1bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:44.836207 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405a9c8-f142-4903-842b-cf9a595d4988 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:44.836503 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766071c-7a47-422b-87bd-d387b12e1bad None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:44.836777 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766071c-7a47-422b-87bd-d387b12e1bad None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:44.837051 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766071c-7a47-422b-87bd-d387b12e1bad None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:44.837700 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766071c-7a47-422b-87bd-d387b12e1bad None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:44.837996 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766071c-7a47-422b-87bd-d387b12e1bad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:44.839379 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766071c-7a47-422b-87bd-d387b12e1bad None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:44.853236 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 503/1032] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:44.859352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766071c-7a47-422b-87bd-d387b12e1bad None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:44.949533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:44.968290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:44.978650 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405a9c8-f142-4903-842b-cf9a595d4988 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:44.996579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766071c-7a47-422b-87bd-d387b12e1bad None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:45.025485 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef7e09-f4a1-4fe7-856b-c05010228c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:45.026125 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef7e09-f4a1-4fe7-856b-c05010228c7c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:45.026400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef7e09-f4a1-4fe7-856b-c05010228c7c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:45.026711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef7e09-f4a1-4fe7-856b-c05010228c7c None admin] PATH_INFO: `/v3/services/image` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:45.027302 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef7e09-f4a1-4fe7-856b-c05010228c7c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:45.028714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef7e09-f4a1-4fe7-856b-c05010228c7c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:45.039116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-2bef7e09-f4a1-4fe7-856b-c05010228c7c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 22:25:45.039116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:45.039116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:45.039116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:45.039116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:45.039116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:45.039116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:45.039116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:45.039116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 22:25:45.039116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:45.039116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:45.039116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:45.039116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:45.039116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:45.039116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:45.039116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:45.039116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:45.040493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:45.040493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:45.040493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:45.040493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:45.040493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:45.040493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:45.040493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:45.040493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:45.040493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:45.040493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:45.040493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:45.040493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:45.040493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:45.040493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:45.040493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:45.040493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:45.041669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:45.041669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:45.041669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:45.041669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:45.041669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:45.041669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:45.041669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:45.041669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:45.041669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:45.041669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:45.041669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:45.041669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:45.041669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 22:25:45.041669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:45.043131 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531c064-c712-4386-b048-36a83774a4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:45.043823 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531c064-c712-4386-b048-36a83774a4dd None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:45.044108 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531c064-c712-4386-b048-36a83774a4dd None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:45.044442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531c064-c712-4386-b048-36a83774a4dd None admin] PATH_INFO: `/v3/users/nova` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:45.045354 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 530/1033] 10.222.0.59 () {56 vars in 1105 bytes} [Mon Aug 29 22:25:44 2022] GET /identity/v3/services/image => generated 86 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:45.046274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef7e09-f4a1-4fe7-856b-c05010228c7c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:45.058677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531c064-c712-4386-b048-36a83774a4dd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:45.060708 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531c064-c712-4386-b048-36a83774a4dd None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:45.074680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-4531c064-c712-4386-b048-36a83774a4dd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:25:45.074680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:45.074680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:45.074680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:45.074680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:45.074680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:45.074680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:45.074680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:45.074680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:45.074680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:45.074680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:45.074680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:45.074680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:25:45.074680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:25:45.074680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:25:45.074680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:25:45.076651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:45.076651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:45.076651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:25:45.076651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:45.076651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:25:45.076651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:45.076651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:45.076651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:45.076651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:45.076651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:45.076651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:45.076651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:45.076651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:45.076651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:45.076651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:45.076651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:45.076651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:45.076651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:45.077691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:45.077691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:45.077691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:25:45.077691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:25:45.077691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:25:45.077691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:25:45.077691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:25:45.077691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:25:45.077691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:25:45.077691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:45.077691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 504/1034] 10.222.0.59 () {56 vars in 1093 bytes} [Mon Aug 29 22:25:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:45.081729 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531c064-c712-4386-b048-36a83774a4dd None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:45.180204 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef7e09-f4a1-4fe7-856b-c05010228c7c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:45.204094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531c064-c712-4386-b048-36a83774a4dd None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:45.222077 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab711cc3-07f1-4f79-a667-d1b220d32e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:45.222800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab711cc3-07f1-4f79-a667-d1b220d32e38 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:45.223074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab711cc3-07f1-4f79-a667-d1b220d32e38 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:45.223430 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab711cc3-07f1-4f79-a667-d1b220d32e38 None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:45.225429 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab711cc3-07f1-4f79-a667-d1b220d32e38 None admin] RBAC: Adding query filter params (name=image) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:45.225748 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab711cc3-07f1-4f79-a667-d1b220d32e38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:45.227145 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab711cc3-07f1-4f79-a667-d1b220d32e38 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:45.236577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 531/1035] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:45.242712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab711cc3-07f1-4f79-a667-d1b220d32e38 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:45.249137 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a6478-b9b4-4020-b8ae-9576816e49f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:45.250125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a6478-b9b4-4020-b8ae-9576816e49f4 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:45.250590 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a6478-b9b4-4020-b8ae-9576816e49f4 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:45.251049 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a6478-b9b4-4020-b8ae-9576816e49f4 None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:45.252338 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a6478-b9b4-4020-b8ae-9576816e49f4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:45.252860 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a6478-b9b4-4020-b8ae-9576816e49f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:45.256081 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a6478-b9b4-4020-b8ae-9576816e49f4 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:45.266929 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:45.291154 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 505/1036] 10.222.0.59 () {56 vars in 1133 bytes} [Mon Aug 29 22:25:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:45.297617 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a6478-b9b4-4020-b8ae-9576816e49f4 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:45.301597 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:45.401567 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab711cc3-07f1-4f79-a667-d1b220d32e38 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:45.423236 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a6478-b9b4-4020-b8ae-9576816e49f4 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:45.447828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f3b3d-c95d-439f-b740-04a24e8bf602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:45.448434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f3b3d-c95d-439f-b740-04a24e8bf602 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:45.448715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f3b3d-c95d-439f-b740-04a24e8bf602 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:45.449009 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f3b3d-c95d-439f-b740-04a24e8bf602 None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:45.449664 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f3b3d-c95d-439f-b740-04a24e8bf602 None admin] RBAC: Adding query filter params (type=image) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:45.450017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f3b3d-c95d-439f-b740-04a24e8bf602 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:45.451476 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f3b3d-c95d-439f-b740-04a24e8bf602 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:45.459377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 532/1037] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:45.460827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c749d-423c-4cb6-915e-02392c61f72c None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:45.461119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c749d-423c-4cb6-915e-02392c61f72c None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:45.461387 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c749d-423c-4cb6-915e-02392c61f72c None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:45.462630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 533/1038] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:45.465030 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c749d-423c-4cb6-915e-02392c61f72c None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:45.480677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca01ca09-f824-4b12-950c-63e1e21faa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:45.481292 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01ca09-f824-4b12-950c-63e1e21faa0a None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:45.481567 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01ca09-f824-4b12-950c-63e1e21faa0a None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:45.481830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01ca09-f824-4b12-950c-63e1e21faa0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:45.554487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:45.559529 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca01ca09-f824-4b12-950c-63e1e21faa0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:45.561000 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca01ca09-f824-4b12-950c-63e1e21faa0a None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:45.563041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:45.585251 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c749d-423c-4cb6-915e-02392c61f72c None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:45.633053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b6bed-d7fc-4f3a-b01c-42a8645df650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:45.633776 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b6bed-d7fc-4f3a-b01c-42a8645df650 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:45.634052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b6bed-d7fc-4f3a-b01c-42a8645df650 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:45.634319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b6bed-d7fc-4f3a-b01c-42a8645df650 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:45.635024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b6bed-d7fc-4f3a-b01c-42a8645df650 None admin] RBAC: Adding query filter params (interface=public, service_id=2cdb90f697e342afad9962e801da54e5, region_id=RegionOne) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:45.635325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b6bed-d7fc-4f3a-b01c-42a8645df650 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:45.636754 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b6bed-d7fc-4f3a-b01c-42a8645df650 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:45.642096 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 534/1039] 10.222.0.59 () {56 vars in 1251 bytes} [Mon Aug 29 22:25:45 2022] GET /identity/v3/endpoints?service_id=2cdb90f697e342afad9962e801da54e5&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:45.646133 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f377982-393e-444c-9829-336d45930406 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:45.646402 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f377982-393e-444c-9829-336d45930406 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:45.646668 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f377982-393e-444c-9829-336d45930406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:45.665534 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-9f377982-393e-444c-9829-336d45930406 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:45.666738 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-9f377982-393e-444c-9829-336d45930406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:45.684100 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 506/1040] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:25:45 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 22:25:45.686088 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca01ca09-f824-4b12-950c-63e1e21faa0a None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:45.742676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 535/1041] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:45.743460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f377982-393e-444c-9829-336d45930406 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:45.809365 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca01ca09-f824-4b12-950c-63e1e21faa0a None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:45.811081 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:45.834011 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:45.852259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59775525-5f78-459a-89bc-e71a6f1174aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:45.852259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59775525-5f78-459a-89bc-e71a6f1174aa None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:45.852259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59775525-5f78-459a-89bc-e71a6f1174aa None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:45.852259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59775525-5f78-459a-89bc-e71a6f1174aa None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:45.852755 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59775525-5f78-459a-89bc-e71a6f1174aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:45.854117 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59775525-5f78-459a-89bc-e71a6f1174aa None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:45.872174 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 507/1042] 10.222.0.59 () {56 vars in 1087 bytes} [Mon Aug 29 22:25:45 2022] GET /identity/v3/services => generated 1957 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:25:45.880678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f377982-393e-444c-9829-336d45930406 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:45.931755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2a72f-70b7-4d7f-895d-2ff9fdf32238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:45.932438 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2a72f-70b7-4d7f-895d-2ff9fdf32238 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:45.932715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2a72f-70b7-4d7f-895d-2ff9fdf32238 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:45.932981 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2a72f-70b7-4d7f-895d-2ff9fdf32238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:45.947822 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2a72f-70b7-4d7f-895d-2ff9fdf32238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:45.949504 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2a72f-70b7-4d7f-895d-2ff9fdf32238 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:45.968780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-6ef2a72f-70b7-4d7f-895d-2ff9fdf32238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:45.968780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:45.968780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:45.968780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:45.968780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:45.968780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:45.968780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:45.968780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:45.968780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:45.968780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:45.968780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:45.968780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:45.968780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:45.968780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:45.968780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:45.968780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:45.969807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:45.969807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:45.969807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:45.969807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:45.969807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:45.969807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:45.969807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:45.969807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:45.969807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:45.969807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:45.969807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:45.969807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:45.969807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:45.969807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:45.969807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:45.969807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:45.969807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:45.976568 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:45.976568 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:45.976568 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:45.976568 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 536/1043] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:45.976904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59775525-5f78-459a-89bc-e71a6f1174aa None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:46.034742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2a72f-70b7-4d7f-895d-2ff9fdf32238 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:46.116210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59775525-5f78-459a-89bc-e71a6f1174aa None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:46.157419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78ea72-90a6-41e1-99d8-856c1334421a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:46.158030 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78ea72-90a6-41e1-99d8-856c1334421a None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:46.158304 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78ea72-90a6-41e1-99d8-856c1334421a None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:46.158568 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78ea72-90a6-41e1-99d8-856c1334421a None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:46.159212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b78ea72-90a6-41e1-99d8-856c1334421a None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:46.159625 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b78ea72-90a6-41e1-99d8-856c1334421a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:46.161546 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b78ea72-90a6-41e1-99d8-856c1334421a None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:46.171997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2a72f-70b7-4d7f-895d-2ff9fdf32238 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:46.178834 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 508/1044] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:46.184569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78ea72-90a6-41e1-99d8-856c1334421a None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:46.198449 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:46.216299 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ef0e7-ad5c-4b6a-84c0-796bdb9aca93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:46.216936 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ef0e7-ad5c-4b6a-84c0-796bdb9aca93 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:46.217207 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ef0e7-ad5c-4b6a-84c0-796bdb9aca93 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:46.217472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ef0e7-ad5c-4b6a-84c0-796bdb9aca93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:46.219085 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:46.230507 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ef0e7-ad5c-4b6a-84c0-796bdb9aca93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:25:46.235464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-128ef0e7-ad5c-4b6a-84c0-796bdb9aca93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:25:46.235464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:46.235464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:46.235464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:46.235464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:46.235464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:46.235464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:46.235464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:46.235464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:46.235464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:46.235464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:46.235464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:46.235464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:25:46.235464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:25:46.235464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:25:46.235464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:25:46.236448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:25:46.236448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:25:46.236448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:25:46.236448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:25:46.236448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:46.236448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:46.236448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:46.236448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:46.236448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:46.236448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:46.236448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:46.236448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:46.236448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:46.236448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:46.236448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:46.236448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:46.237297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:46.237297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:46.237297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:46.237297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:46.237297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:25:46.237297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:25:46.237297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:25:46.237297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:25:46.237297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:25:46.237297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:25:46.237297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:25:46.237297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:46.237297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 537/1045] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:25:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:46.245027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ef0e7-ad5c-4b6a-84c0-796bdb9aca93 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:46.314130 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78ea72-90a6-41e1-99d8-856c1334421a None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:46.363482 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb4e98-b59a-4649-989f-f915353ad951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:46.364830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb4e98-b59a-4649-989f-f915353ad951 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:46.365598 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb4e98-b59a-4649-989f-f915353ad951 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:46.366335 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb4e98-b59a-4649-989f-f915353ad951 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:46.375554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb4e98-b59a-4649-989f-f915353ad951 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:46.375776 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ef0e7-ad5c-4b6a-84c0-796bdb9aca93 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:46.377669 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb4e98-b59a-4649-989f-f915353ad951 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:46.392466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-8ebb4e98-b59a-4649-989f-f915353ad951 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:46.392466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:46.392466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:46.392466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:46.392466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:46.392466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:46.392466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:46.392466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:46.392466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:46.392466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:46.392466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:46.392466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:46.392466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:25:46.392466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:25:46.392466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:25:46.393508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:25:46.393508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:46.393508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:46.393508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:46.393508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:46.393508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:46.393508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:46.393508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:46.393508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:46.393508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:46.393508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:46.393508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:46.393508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:46.393508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:46.393508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:46.393508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:46.393508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:46.394472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:25:46.394472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:25:46.394472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:25:46.394472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:25:46.394472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:25:46.394472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:25:46.394472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:46.394472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:46.396206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 509/1046] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:46.403823 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb4e98-b59a-4649-989f-f915353ad951 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:46.419117 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a1256-753f-42ae-affe-58549830a227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:46.419868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a1256-753f-42ae-affe-58549830a227 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:46.420192 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a1256-753f-42ae-affe-58549830a227 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:46.420538 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a1256-753f-42ae-affe-58549830a227 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:46.421910 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a1256-753f-42ae-affe-58549830a227 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:46.422269 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a1256-753f-42ae-affe-58549830a227 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:46.423905 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a1256-753f-42ae-affe-58549830a227 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:46.434186 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 538/1047] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:25:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:46.440742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a1256-753f-42ae-affe-58549830a227 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:46.496862 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:46.515282 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:46.533730 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb4e98-b59a-4649-989f-f915353ad951 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:46.564604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a1256-753f-42ae-affe-58549830a227 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:46.578882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2d578-ad82-428b-b422-bbcf32b447ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:46.579872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2d578-ad82-428b-b422-bbcf32b447ad None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:46.580379 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2d578-ad82-428b-b422-bbcf32b447ad None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:46.580980 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2d578-ad82-428b-b422-bbcf32b447ad None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:46.582098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2d578-ad82-428b-b422-bbcf32b447ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:46.582612 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2d578-ad82-428b-b422-bbcf32b447ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:46.585077 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2d578-ad82-428b-b422-bbcf32b447ad None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:46.600673 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 510/1048] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:25:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:46.606815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2d578-ad82-428b-b422-bbcf32b447ad None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:46.610973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987d883-ab85-4351-8756-c15590be68f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:46.611615 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987d883-ab85-4351-8756-c15590be68f9 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:46.611890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987d883-ab85-4351-8756-c15590be68f9 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:46.612183 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987d883-ab85-4351-8756-c15590be68f9 None admin] PATH_INFO: `/v3/users/ce26a12c29d54d948ad17cb664229971` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:46.627835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987d883-ab85-4351-8756-c15590be68f9 None admin] RBAC: Authorizing `identity:get_user(user_id=ce26a12c29d54d948ad17cb664229971)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:46.629749 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987d883-ab85-4351-8756-c15590be68f9 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:46.631939 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 539/1049] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:25:46 2022] GET /identity/v3/users/ce26a12c29d54d948ad17cb664229971 => generated 285 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:46.638204 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987d883-ab85-4351-8756-c15590be68f9 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:46.737140 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2d578-ad82-428b-b422-bbcf32b447ad None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:46.763646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:46.766563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987d883-ab85-4351-8756-c15590be68f9 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:46.788224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac812fc1-d2f6-4b3c-a05c-f1388b8c93c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:46.788937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac812fc1-d2f6-4b3c-a05c-f1388b8c93c3 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:46.789225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac812fc1-d2f6-4b3c-a05c-f1388b8c93c3 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:46.789494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac812fc1-d2f6-4b3c-a05c-f1388b8c93c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:46.792051 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac812fc1-d2f6-4b3c-a05c-f1388b8c93c3 None admin] RBAC: Adding query filter params (role.id=04230301829e46deb6a9a0f33b868e35, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=7f21105679f245a19ea4850e6c010dbf) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:46.792051 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac812fc1-d2f6-4b3c-a05c-f1388b8c93c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:46.792948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac812fc1-d2f6-4b3c-a05c-f1388b8c93c3 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:46.800435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 511/1050] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:25:46 2022] GET /identity/v3/role_assignments?user.id=7f21105679f245a19ea4850e6c010dbf&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=04230301829e46deb6a9a0f33b868e35 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:46.815365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8fd83f-65d0-4689-8ae1-f4fb3617e960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:46.816017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fd83f-65d0-4689-8ae1-f4fb3617e960 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:46.816384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fd83f-65d0-4689-8ae1-f4fb3617e960 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:46.816716 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fd83f-65d0-4689-8ae1-f4fb3617e960 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:46.919371 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8fd83f-65d0-4689-8ae1-f4fb3617e960 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:46.923649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8fd83f-65d0-4689-8ae1-f4fb3617e960 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:47.055784 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:47.065118 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 540/1051] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:25:46 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 22:25:47.070058 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac812fc1-d2f6-4b3c-a05c-f1388b8c93c3 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:47.084430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:47.202287 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac812fc1-d2f6-4b3c-a05c-f1388b8c93c3 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:47.246714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b59bd-d737-4851-a8a8-9ef8dcbf8533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:47.247643 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b59bd-d737-4851-a8a8-9ef8dcbf8533 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:47.248090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b59bd-d737-4851-a8a8-9ef8dcbf8533 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:47.248576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b59bd-d737-4851-a8a8-9ef8dcbf8533 None admin] PATH_INFO: `/v3/projects/2ba6be0a6c3e47bcafd8d8be0d24ea1e` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:47.249946 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573b59bd-d737-4851-a8a8-9ef8dcbf8533 None admin] RBAC: Authorizing `identity:get_project(project_id=2ba6be0a6c3e47bcafd8d8be0d24ea1e)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:47.251530 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573b59bd-d737-4851-a8a8-9ef8dcbf8533 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:47.253426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 512/1052] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:25:47 2022] GET /identity/v3/projects/2ba6be0a6c3e47bcafd8d8be0d24ea1e => generated 311 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:47.261331 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8fd83f-65d0-4689-8ae1-f4fb3617e960 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:47.392995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8fd83f-65d0-4689-8ae1-f4fb3617e960 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:47.433593 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce7f51-7341-469c-9c99-42c6f9bfc819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:47.434249 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce7f51-7341-469c-9c99-42c6f9bfc819 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:47.434481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce7f51-7341-469c-9c99-42c6f9bfc819 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:47.434779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce7f51-7341-469c-9c99-42c6f9bfc819 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:47.435507 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce7f51-7341-469c-9c99-42c6f9bfc819 None admin] RBAC: Adding query filter params (role.id=a836108447dc439e994611bc74d53d9c, scope.project.id=2ba6be0a6c3e47bcafd8d8be0d24ea1e, user.id=ce26a12c29d54d948ad17cb664229971) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:47.435806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce7f51-7341-469c-9c99-42c6f9bfc819 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:47.437275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce7f51-7341-469c-9c99-42c6f9bfc819 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:47.443413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 541/1053] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:25:47 2022] GET /identity/v3/role_assignments?user.id=ce26a12c29d54d948ad17cb664229971&scope.project.id=2ba6be0a6c3e47bcafd8d8be0d24ea1e&role.id=a836108447dc439e994611bc74d53d9c => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:47.894790 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e50f6f-28ff-4c08-93fd-89e4434799c0 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:47.895154 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e50f6f-28ff-4c08-93fd-89e4434799c0 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:47.895957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e50f6f-28ff-4c08-93fd-89e4434799c0 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:47.897830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 513/1054] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:47.903545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca879bd-7076-4d52-8cfe-776973fd3f1d None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:47.903952 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca879bd-7076-4d52-8cfe-776973fd3f1d None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:47.904161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca879bd-7076-4d52-8cfe-776973fd3f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:47.929162 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-cca879bd-7076-4d52-8cfe-776973fd3f1d None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:47.930853 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-cca879bd-7076-4d52-8cfe-776973fd3f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:48.000488 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:48.012179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 542/1055] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:48.288740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e50f6f-28ff-4c08-93fd-89e4434799c0 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:48.387157 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:48.416787 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e50f6f-28ff-4c08-93fd-89e4434799c0 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:48.455938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fabec7a-70da-43cc-b7fa-440f70352912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:48.456608 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabec7a-70da-43cc-b7fa-440f70352912 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:48.456878 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabec7a-70da-43cc-b7fa-440f70352912 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:48.457149 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabec7a-70da-43cc-b7fa-440f70352912 None admin] PATH_INFO: `/v3/services/image` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:48.457778 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fabec7a-70da-43cc-b7fa-440f70352912 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:48.459134 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fabec7a-70da-43cc-b7fa-440f70352912 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:48.468886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-5fabec7a-70da-43cc-b7fa-440f70352912 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 22:25:48.468886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:48.468886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:25:48.468886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:25:48.468886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:25:48.468886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:25:48.468886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:25:48.468886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:48.468886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 22:25:48.468886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:48.468886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:25:48.468886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:48.468886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:48.468886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:48.468886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:48.468886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:48.468886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:48.470056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:48.470056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:48.470056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:48.470056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:48.470056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:48.470056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:48.470056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:25:48.470056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:25:48.470056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:25:48.470056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:25:48.470056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:48.470056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:48.470056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:48.470056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:48.470056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:48.470056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:48.470970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:48.470970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:48.470970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:48.470970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:48.470970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:48.470970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:48.470970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:48.470970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:48.470970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:48.470970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:48.470970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:25:48.470970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:25:48.470970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 22:25:48.470970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:48.470970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 514/1056] 10.222.0.59 () {56 vars in 1105 bytes} [Mon Aug 29 22:25:48 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:48.480256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca879bd-7076-4d52-8cfe-776973fd3f1d None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:48.608386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca879bd-7076-4d52-8cfe-776973fd3f1d None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:48.649333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb207907-0875-492d-be83-718c7b3ffe2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:48.649944 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb207907-0875-492d-be83-718c7b3ffe2b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:48.650218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb207907-0875-492d-be83-718c7b3ffe2b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:48.650483 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb207907-0875-492d-be83-718c7b3ffe2b None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:48.651109 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb207907-0875-492d-be83-718c7b3ffe2b None admin] RBAC: Adding query filter params (name=image) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:48.651402 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb207907-0875-492d-be83-718c7b3ffe2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:48.654297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb207907-0875-492d-be83-718c7b3ffe2b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:48.665343 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 543/1057] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:48.667155 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fabec7a-70da-43cc-b7fa-440f70352912 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:48.784706 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a753fb0-3444-492f-b7f8-5818acc95e9a None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:48.785049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a753fb0-3444-492f-b7f8-5818acc95e9a None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:48.785312 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a753fb0-3444-492f-b7f8-5818acc95e9a None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:48.787318 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 544/1058] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:25:48.792722 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a06c2f-856b-4e3b-a604-687c5a1ad627 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:48.793055 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a06c2f-856b-4e3b-a604-687c5a1ad627 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:48.793281 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a06c2f-856b-4e3b-a604-687c5a1ad627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:48.811854 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fabec7a-70da-43cc-b7fa-440f70352912 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:48.814082 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-a1a06c2f-856b-4e3b-a604-687c5a1ad627 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:48.815545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a1a06c2f-856b-4e3b-a604-687c5a1ad627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:48.853769 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01843f65-74f4-45a8-b7f7-8e27039f0f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:48.854536 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01843f65-74f4-45a8-b7f7-8e27039f0f04 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:48.854810 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01843f65-74f4-45a8-b7f7-8e27039f0f04 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:48.855083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01843f65-74f4-45a8-b7f7-8e27039f0f04 None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:48.855773 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01843f65-74f4-45a8-b7f7-8e27039f0f04 None admin] RBAC: Adding query filter params (type=image) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:48.856013 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01843f65-74f4-45a8-b7f7-8e27039f0f04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:48.857399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01843f65-74f4-45a8-b7f7-8e27039f0f04 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:48.863194 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 515/1059] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:48 2022] GET /identity/v3/services?type=image => generated 350 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:48.872372 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01843f65-74f4-45a8-b7f7-8e27039f0f04 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:48.882847 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:48.888306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:48.905738 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 545/1060] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:48 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:49.003610 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01843f65-74f4-45a8-b7f7-8e27039f0f04 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:49.047763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a4908-8a61-4201-86af-0f0cd8307ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:49.048410 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a4908-8a61-4201-86af-0f0cd8307ae9 None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:49.048682 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a4908-8a61-4201-86af-0f0cd8307ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:49.048944 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a4908-8a61-4201-86af-0f0cd8307ae9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:49.049602 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a4908-8a61-4201-86af-0f0cd8307ae9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:49.050946 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a4908-8a61-4201-86af-0f0cd8307ae9 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:49.136719 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 516/1061] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:25:48 2022] POST /identity/v3/endpoints => generated 327 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:25:49.188057 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a06c2f-856b-4e3b-a604-687c5a1ad627 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:49.330448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a06c2f-856b-4e3b-a604-687c5a1ad627 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:49.381259 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c1a7f-c904-4cfe-b08b-b7aa724c2597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:49.382386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c1a7f-c904-4cfe-b08b-b7aa724c2597 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:49.382386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c1a7f-c904-4cfe-b08b-b7aa724c2597 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:49.382559 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c1a7f-c904-4cfe-b08b-b7aa724c2597 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:49.396230 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c1a7f-c904-4cfe-b08b-b7aa724c2597 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:25:49.402171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-7c4c1a7f-c904-4cfe-b08b-b7aa724c2597 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:25:49.402171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:49.402171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:49.402171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:49.402171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:49.402171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:49.402171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:49.402171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:49.402171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:49.402171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:49.402171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:49.402171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:49.402171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:25:49.402171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:25:49.402171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:25:49.402171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:25:49.403335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:25:49.403335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:25:49.403335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:25:49.403335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:25:49.403335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:49.403335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:49.403335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:49.403335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:49.403335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:49.403335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:49.403335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:49.403335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:49.403335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:49.403335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:49.403335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:49.403335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:49.404377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:49.404377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:49.404377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:49.404377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:49.404377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:25:49.404377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:25:49.404377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:25:49.404377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:25:49.404377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:25:49.404377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:25:49.404377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:25:49.404377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:49.404377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 546/1062] 10.222.0.59 () {56 vars in 1120 bytes} [Mon Aug 29 22:25:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:49.410420 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a4908-8a61-4201-86af-0f0cd8307ae9 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:49.415831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9f779-279c-4d2b-8ec8-9268e9cf9826 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:49.416174 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9f779-279c-4d2b-8ec8-9268e9cf9826 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:49.416405 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9f779-279c-4d2b-8ec8-9268e9cf9826 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:49.417913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 547/1063] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:25:49.423391 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d04cb2-9789-4b14-bf0a-46cc345ee854 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:49.423676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d04cb2-9789-4b14-bf0a-46cc345ee854 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:49.423954 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d04cb2-9789-4b14-bf0a-46cc345ee854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:49.445034 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-e7d04cb2-9789-4b14-bf0a-46cc345ee854 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:49.446245 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e7d04cb2-9789-4b14-bf0a-46cc345ee854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:49.489418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:49.494896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:49.531073 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 548/1064] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:49.543566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a4908-8a61-4201-86af-0f0cd8307ae9 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:49.585148 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6bbf2-f74b-4cf4-91c6-06ab1bd8345c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:49.585945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6bbf2-f74b-4cf4-91c6-06ab1bd8345c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:49.586224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6bbf2-f74b-4cf4-91c6-06ab1bd8345c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:49.586490 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6bbf2-f74b-4cf4-91c6-06ab1bd8345c None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:49.587162 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe6bbf2-f74b-4cf4-91c6-06ab1bd8345c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:49.587458 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe6bbf2-f74b-4cf4-91c6-06ab1bd8345c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:49.588858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe6bbf2-f74b-4cf4-91c6-06ab1bd8345c None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:49.600952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 517/1065] 10.222.0.59 () {56 vars in 1115 bytes} [Mon Aug 29 22:25:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:49.606561 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d04cb2-9789-4b14-bf0a-46cc345ee854 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:49.728865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d04cb2-9789-4b14-bf0a-46cc345ee854 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:49.774569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6305109-7cc0-4d73-bfe2-b975299c17b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:49.775209 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6305109-7cc0-4d73-bfe2-b975299c17b7 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:49.775486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6305109-7cc0-4d73-bfe2-b975299c17b7 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:49.775757 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6305109-7cc0-4d73-bfe2-b975299c17b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:49.787582 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6305109-7cc0-4d73-bfe2-b975299c17b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:49.789108 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6305109-7cc0-4d73-bfe2-b975299c17b7 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:49.803267 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6bbf2-f74b-4cf4-91c6-06ab1bd8345c None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:49.807218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-e6305109-7cc0-4d73-bfe2-b975299c17b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:49.807218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:49.807218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:49.807218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:49.807218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:49.807218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:49.807218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:49.807218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:49.807218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:49.807218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:49.807218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:49.807218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:49.807218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:49.807218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:49.807218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:49.807218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:49.808511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:49.808511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:49.808511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:49.808511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:49.808511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:49.808511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:49.808511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:49.808511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:49.808511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:49.808511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:49.808511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:49.808511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:49.808511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:49.808511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:49.808511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:49.808511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:49.808511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:49.809534 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:49.809534 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:49.809534 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:49.813633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 549/1066] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:49.819678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6305109-7cc0-4d73-bfe2-b975299c17b7 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:49.859398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:49.941170 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6bbf2-f74b-4cf4-91c6-06ab1bd8345c None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:49.956888 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6305109-7cc0-4d73-bfe2-b975299c17b7 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:49.968161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:49.979989 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc5e84-6a9e-418f-b104-577d1010346d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:49.980573 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc5e84-6a9e-418f-b104-577d1010346d None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:49.980847 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc5e84-6a9e-418f-b104-577d1010346d None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:49.981111 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc5e84-6a9e-418f-b104-577d1010346d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:49.995477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc5e84-6a9e-418f-b104-577d1010346d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:25:50.000580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-0ddc5e84-6a9e-418f-b104-577d1010346d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:25:50.000580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:50.000580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:50.000580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:50.000580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:50.000580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:50.000580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:50.000580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:50.000580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:50.000580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:50.000580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:50.000580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:50.000580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:25:50.000580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:25:50.000580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:25:50.000580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:25:50.001651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:25:50.001651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:25:50.001651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:25:50.001651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:25:50.001651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:50.001651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:50.001651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:50.001651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:50.001651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:50.001651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:50.001651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:50.001651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:50.001651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:50.001651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:50.001651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:50.001651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:50.002605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:50.002605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:50.002605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:50.002605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:50.002605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:25:50.002605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:25:50.002605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:25:50.002605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:25:50.002605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:25:50.002605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:25:50.002605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:25:50.002605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:50.005881 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327767bb-ec2e-4f7d-b25a-4775cb70b089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:50.007079 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 518/1067] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:25:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:50.007282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327767bb-ec2e-4f7d-b25a-4775cb70b089 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:50.007719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327767bb-ec2e-4f7d-b25a-4775cb70b089 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:50.007885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc5e84-6a9e-418f-b104-577d1010346d None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:50.008368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327767bb-ec2e-4f7d-b25a-4775cb70b089 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:50.009163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327767bb-ec2e-4f7d-b25a-4775cb70b089 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:50.009634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327767bb-ec2e-4f7d-b25a-4775cb70b089 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:50.011032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327767bb-ec2e-4f7d-b25a-4775cb70b089 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:50.029317 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 550/1068] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:50.030209 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327767bb-ec2e-4f7d-b25a-4775cb70b089 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:50.117936 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc5e84-6a9e-418f-b104-577d1010346d None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:50.134774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:50.151854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327767bb-ec2e-4f7d-b25a-4775cb70b089 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:50.165142 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05c3ce-981c-40be-80b8-0fc3a9a13732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:50.165788 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05c3ce-981c-40be-80b8-0fc3a9a13732 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:50.166064 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05c3ce-981c-40be-80b8-0fc3a9a13732 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:50.166345 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05c3ce-981c-40be-80b8-0fc3a9a13732 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:50.166994 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05c3ce-981c-40be-80b8-0fc3a9a13732 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:50.167292 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05c3ce-981c-40be-80b8-0fc3a9a13732 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:50.168748 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05c3ce-981c-40be-80b8-0fc3a9a13732 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:50.180359 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 519/1069] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:25:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:50.185888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05c3ce-981c-40be-80b8-0fc3a9a13732 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:50.195554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861a785-54cf-4bfc-82a1-c594cc6f9986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:50.196223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861a785-54cf-4bfc-82a1-c594cc6f9986 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:50.196505 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861a785-54cf-4bfc-82a1-c594cc6f9986 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:50.196858 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861a785-54cf-4bfc-82a1-c594cc6f9986 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:50.207020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861a785-54cf-4bfc-82a1-c594cc6f9986 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:50.210483 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861a785-54cf-4bfc-82a1-c594cc6f9986 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:50.227610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-5861a785-54cf-4bfc-82a1-c594cc6f9986 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:25:50.227610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:50.227610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:50.227610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:50.227610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:50.227610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:50.227610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:50.227610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:50.227610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:50.227610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:50.227610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:50.227610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:50.227610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:25:50.227610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:25:50.227610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:25:50.227610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:25:50.228696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:50.228696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:50.228696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:25:50.228696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:50.228696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:25:50.228696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:50.228696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:50.228696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:50.228696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:50.228696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:50.228696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:50.228696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:50.228696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:50.228696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:50.228696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:50.228696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:50.228696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:50.228696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:50.237267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:50.237267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:50.237267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:25:50.237267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:25:50.237267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:25:50.237267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:25:50.237267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:25:50.237267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:25:50.237267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:25:50.237267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:50.237267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 551/1070] 10.222.0.59 () {56 vars in 1093 bytes} [Mon Aug 29 22:25:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:50.237267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861a785-54cf-4bfc-82a1-c594cc6f9986 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:50.245225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:50.313575 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05c3ce-981c-40be-80b8-0fc3a9a13732 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:50.355195 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc2da7-4579-45cd-b358-e1609d1374fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:50.355836 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc2da7-4579-45cd-b358-e1609d1374fb None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:50.356111 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc2da7-4579-45cd-b358-e1609d1374fb None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:50.356405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc2da7-4579-45cd-b358-e1609d1374fb None admin] PATH_INFO: `/v3/users/ce26a12c29d54d948ad17cb664229971` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:50.356837 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861a785-54cf-4bfc-82a1-c594cc6f9986 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:50.357641 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc2da7-4579-45cd-b358-e1609d1374fb None admin] RBAC: Authorizing `identity:get_user(user_id=ce26a12c29d54d948ad17cb664229971)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:50.359049 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc2da7-4579-45cd-b358-e1609d1374fb None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:50.360828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 520/1071] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:25:50 2022] GET /identity/v3/users/ce26a12c29d54d948ad17cb664229971 => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:50.366964 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc2da7-4579-45cd-b358-e1609d1374fb None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:50.403395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b01acc-27d8-4b8d-b447-dd9fbbc0a51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:50.404347 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b01acc-27d8-4b8d-b447-dd9fbbc0a51c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:50.404614 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b01acc-27d8-4b8d-b447-dd9fbbc0a51c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:50.404883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b01acc-27d8-4b8d-b447-dd9fbbc0a51c None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:50.405618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b01acc-27d8-4b8d-b447-dd9fbbc0a51c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:50.405913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b01acc-27d8-4b8d-b447-dd9fbbc0a51c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:50.407437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b01acc-27d8-4b8d-b447-dd9fbbc0a51c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:50.439044 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 552/1072] 10.222.0.59 () {56 vars in 1133 bytes} [Mon Aug 29 22:25:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:50.444643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b01acc-27d8-4b8d-b447-dd9fbbc0a51c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:50.448450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:50.497396 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc2da7-4579-45cd-b358-e1609d1374fb None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:50.515275 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:50.537706 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c06c8-6a4a-470d-8d5d-682a32817573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:50.538348 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c06c8-6a4a-470d-8d5d-682a32817573 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:50.538921 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c06c8-6a4a-470d-8d5d-682a32817573 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:50.539192 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c06c8-6a4a-470d-8d5d-682a32817573 None admin] PATH_INFO: `/v3/projects/2ba6be0a6c3e47bcafd8d8be0d24ea1e` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:50.540707 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c06c8-6a4a-470d-8d5d-682a32817573 None admin] RBAC: Authorizing `identity:get_project(project_id=2ba6be0a6c3e47bcafd8d8be0d24ea1e)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:50.542286 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c06c8-6a4a-470d-8d5d-682a32817573 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:50.544606 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 521/1073] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:25:50 2022] GET /identity/v3/projects/2ba6be0a6c3e47bcafd8d8be0d24ea1e => generated 311 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:50.550211 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c06c8-6a4a-470d-8d5d-682a32817573 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:50.557190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b01acc-27d8-4b8d-b447-dd9fbbc0a51c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:50.619372 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ceb29-12b4-4e87-8ee5-f86baca91be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:50.620399 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ceb29-12b4-4e87-8ee5-f86baca91be3 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:50.620850 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ceb29-12b4-4e87-8ee5-f86baca91be3 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:50.621267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ceb29-12b4-4e87-8ee5-f86baca91be3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:50.635121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ceb29-12b4-4e87-8ee5-f86baca91be3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:50.636679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ceb29-12b4-4e87-8ee5-f86baca91be3 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:50.652025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-623ceb29-12b4-4e87-8ee5-f86baca91be3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:50.652025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:50.652025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:50.652025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:50.652025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:50.652025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:50.652025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:50.652025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:50.652025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:50.652025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:50.652025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:50.652025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:50.652025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:50.652025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:50.652025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:50.652025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:50.653159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:50.653159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:50.653159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:50.653159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:50.653159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:50.653159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:50.653159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:50.653159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:50.653159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:50.653159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:50.653159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:50.653159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:50.653159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:50.653159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:50.653159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:50.653159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:50.653159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:50.657302 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:50.657302 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:50.657302 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:50.657302 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 553/1074] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:50.659599 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ceb29-12b4-4e87-8ee5-f86baca91be3 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:50.668629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c06c8-6a4a-470d-8d5d-682a32817573 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:50.718081 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3476de7-c4c0-4204-ad06-2b891221a954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:50.719119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3476de7-c4c0-4204-ad06-2b891221a954 None admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:50.719544 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3476de7-c4c0-4204-ad06-2b891221a954 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:50.719995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3476de7-c4c0-4204-ad06-2b891221a954 None admin] PATH_INFO: `/v3/projects/2ba6be0a6c3e47bcafd8d8be0d24ea1e/users/ce26a12c29d54d948ad17cb664229971/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:50.724199 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3476de7-c4c0-4204-ad06-2b891221a954 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ba6be0a6c3e47bcafd8d8be0d24ea1e, user_id=ce26a12c29d54d948ad17cb664229971, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:50.725761 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3476de7-c4c0-4204-ad06-2b891221a954 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:50.735893 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:50.779924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ceb29-12b4-4e87-8ee5-f86baca91be3 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:50.815423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6281de-3f3f-43c9-a6e8-97713cd14952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:50.816022 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6281de-3f3f-43c9-a6e8-97713cd14952 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:50.816332 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6281de-3f3f-43c9-a6e8-97713cd14952 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:50.816595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6281de-3f3f-43c9-a6e8-97713cd14952 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:50.817257 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6281de-3f3f-43c9-a6e8-97713cd14952 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:50.817550 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6281de-3f3f-43c9-a6e8-97713cd14952 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:50.818902 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6281de-3f3f-43c9-a6e8-97713cd14952 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:50.830611 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 554/1075] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:50.836484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6281de-3f3f-43c9-a6e8-97713cd14952 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:50.839556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 522/1076] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:25:50 2022] PUT /identity/v3/projects/2ba6be0a6c3e47bcafd8d8be0d24ea1e/users/ce26a12c29d54d948ad17cb664229971/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:25:50.979636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6281de-3f3f-43c9-a6e8-97713cd14952 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:51.001466 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:51.024238 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc0122-f18e-4fdb-ba2d-35931fb2caf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:51.025247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc0122-f18e-4fdb-ba2d-35931fb2caf9 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:51.025518 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc0122-f18e-4fdb-ba2d-35931fb2caf9 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:51.025784 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc0122-f18e-4fdb-ba2d-35931fb2caf9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:51.033095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc0122-f18e-4fdb-ba2d-35931fb2caf9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:51.034456 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc0122-f18e-4fdb-ba2d-35931fb2caf9 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:51.047532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-2ccc0122-f18e-4fdb-ba2d-35931fb2caf9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:51.047532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:51.047532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:51.047532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:51.047532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:51.047532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:51.047532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:51.047532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:51.047532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:51.047532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:51.047532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:51.047532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:51.047532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:25:51.047532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:25:51.047532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:25:51.048768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:25:51.048768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:51.048768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:51.048768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:51.048768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:51.048768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:51.048768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:51.048768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:51.048768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:51.048768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:51.048768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:51.048768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:51.048768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:51.048768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:51.048768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:51.048768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:51.048768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:51.050418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:25:51.050418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:25:51.050418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:25:51.050418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:25:51.050418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:25:51.050418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:25:51.050418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:51.050418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:51.050418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 555/1077] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:51.055104 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3476de7-c4c0-4204-ad06-2b891221a954 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:51.104289 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:51.164385 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3476de7-c4c0-4204-ad06-2b891221a954 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:51.204274 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12396f28-0aae-4c31-833f-6b95d7d3b06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:51.210134 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12396f28-0aae-4c31-833f-6b95d7d3b06f None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:51.210134 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12396f28-0aae-4c31-833f-6b95d7d3b06f None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:51.210134 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12396f28-0aae-4c31-833f-6b95d7d3b06f None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:51.210134 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12396f28-0aae-4c31-833f-6b95d7d3b06f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:51.210134 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12396f28-0aae-4c31-833f-6b95d7d3b06f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:51.210698 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf9791-fbd8-4af7-8448-d7e83f9cbf3a None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:51.210791 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12396f28-0aae-4c31-833f-6b95d7d3b06f None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:51.210883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf9791-fbd8-4af7-8448-d7e83f9cbf3a None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:51.210883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf9791-fbd8-4af7-8448-d7e83f9cbf3a None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:51.210883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 556/1078] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:25:51.215706 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a3182-b64e-45b2-8d0f-794187f228c1 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:51.215985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a3182-b64e-45b2-8d0f-794187f228c1 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:51.216297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a3182-b64e-45b2-8d0f-794187f228c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:51.229089 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 523/1079] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:25:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:51.235087 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12396f28-0aae-4c31-833f-6b95d7d3b06f None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:51.235808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-599a3182-b64e-45b2-8d0f-794187f228c1 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:51.237041 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-599a3182-b64e-45b2-8d0f-794187f228c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:51.330739 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 557/1080] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:51.359141 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12396f28-0aae-4c31-833f-6b95d7d3b06f None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:51.360798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:51.401763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac6181-0105-457a-b8f5-94000f2aadff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:51.402428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac6181-0105-457a-b8f5-94000f2aadff None admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:51.402701 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac6181-0105-457a-b8f5-94000f2aadff None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:51.402969 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac6181-0105-457a-b8f5-94000f2aadff None admin] PATH_INFO: `/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/7f21105679f245a19ea4850e6c010dbf/roles/04230301829e46deb6a9a0f33b868e35` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:51.405263 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac6181-0105-457a-b8f5-94000f2aadff None admin] RBAC: Authorizing `identity:create_grant(project_id=7799820fc7c64fd6a99a0a9ccf1338e3, user_id=7f21105679f245a19ea4850e6c010dbf, role_id=04230301829e46deb6a9a0f33b868e35)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:51.406667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac6181-0105-457a-b8f5-94000f2aadff None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:51.508802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 524/1081] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:25:51 2022] PUT /identity/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/7f21105679f245a19ea4850e6c010dbf/roles/04230301829e46deb6a9a0f33b868e35 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:25:51.618121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a3182-b64e-45b2-8d0f-794187f228c1 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:51.735675 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a3182-b64e-45b2-8d0f-794187f228c1 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:51.743708 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:51.771047 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd5221-214d-4f13-8d41-d1ba21279a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:51.771645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd5221-214d-4f13-8d41-d1ba21279a62 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:51.771914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd5221-214d-4f13-8d41-d1ba21279a62 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:51.772210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd5221-214d-4f13-8d41-d1ba21279a62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:51.846956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd5221-214d-4f13-8d41-d1ba21279a62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:51.849495 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd5221-214d-4f13-8d41-d1ba21279a62 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:51.966177 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 558/1082] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:25:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:25:51.972640 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac6181-0105-457a-b8f5-94000f2aadff None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:52.081542 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac6181-0105-457a-b8f5-94000f2aadff None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:52.119272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088aab1e-cf52-4801-850a-89fd5b385c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:52.119905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088aab1e-cf52-4801-850a-89fd5b385c77 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:52.120199 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088aab1e-cf52-4801-850a-89fd5b385c77 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:52.120471 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088aab1e-cf52-4801-850a-89fd5b385c77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:52.131793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088aab1e-cf52-4801-850a-89fd5b385c77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:52.133854 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088aab1e-cf52-4801-850a-89fd5b385c77 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:52.149293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-088aab1e-cf52-4801-850a-89fd5b385c77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:52.149293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:52.149293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:52.149293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:52.149293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:52.149293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:52.149293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:52.149293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:52.149293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:52.149293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:52.149293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:52.149293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:52.149293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:52.149293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:52.149293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:52.149293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:52.150327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:52.150327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:52.150327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:52.150327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:52.150327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:52.150327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:52.150327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:52.150327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:52.150327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:52.150327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:52.150327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:52.150327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:52.150327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:52.150327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:52.150327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:52.150327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:52.150327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:52.151403 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:52.151403 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:52.151403 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:52.151403 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 525/1083] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:52.158433 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd5221-214d-4f13-8d41-d1ba21279a62 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:52.166496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:52.294197 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd5221-214d-4f13-8d41-d1ba21279a62 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:52.331023 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e15c2e-4c12-4926-82c0-a1f71d88143e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:52.331622 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e15c2e-4c12-4926-82c0-a1f71d88143e None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:52.331902 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e15c2e-4c12-4926-82c0-a1f71d88143e None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:52.332213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e15c2e-4c12-4926-82c0-a1f71d88143e None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:52.333682 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e15c2e-4c12-4926-82c0-a1f71d88143e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:52.333988 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e15c2e-4c12-4926-82c0-a1f71d88143e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:52.335458 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e15c2e-4c12-4926-82c0-a1f71d88143e None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:52.347475 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 559/1084] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:52.823961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68fb26-7d06-4070-988d-9fe52c796c9d None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:52.824698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68fb26-7d06-4070-988d-9fe52c796c9d None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:52.824698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68fb26-7d06-4070-988d-9fe52c796c9d None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:52.825853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 526/1085] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:52.831330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789960f6-f7dd-49d1-ae0a-72d6179928d8 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:52.831858 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789960f6-f7dd-49d1-ae0a-72d6179928d8 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:52.831975 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789960f6-f7dd-49d1-ae0a-72d6179928d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:52.858475 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-789960f6-f7dd-49d1-ae0a-72d6179928d8 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:52.859674 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-789960f6-f7dd-49d1-ae0a-72d6179928d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:52.945511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:52.953123 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 560/1086] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:53.230312 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68fb26-7d06-4070-988d-9fe52c796c9d None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:53.248580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:53.337749 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68fb26-7d06-4070-988d-9fe52c796c9d None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:53.372321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1878935-97e2-4f15-a71d-ad81c847532f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:53.372940 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1878935-97e2-4f15-a71d-ad81c847532f None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:53.373214 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1878935-97e2-4f15-a71d-ad81c847532f None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:53.373499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1878935-97e2-4f15-a71d-ad81c847532f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:53.384881 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1878935-97e2-4f15-a71d-ad81c847532f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:25:53.389690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-c1878935-97e2-4f15-a71d-ad81c847532f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:25:53.389690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:53.389690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:53.389690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:53.389690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:53.389690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:53.389690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:53.389690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:53.389690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:53.389690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:53.389690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:53.389690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:53.389690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:25:53.389690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:25:53.389690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:25:53.389690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:25:53.390819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:25:53.390819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:25:53.390819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:25:53.390819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:25:53.390819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:53.390819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:53.390819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:53.390819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:53.390819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:53.390819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:53.390819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:53.390819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:53.390819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:53.390819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:53.390819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:53.390819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:53.393107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:53.393107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:53.393107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:53.393107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:53.393107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:25:53.393107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:25:53.393107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:25:53.393107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:25:53.393107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:25:53.393107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:25:53.393107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:25:53.393107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:53.393107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 527/1087] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:25:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:53.397849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789960f6-f7dd-49d1-ae0a-72d6179928d8 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:53.496398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ed156-fb96-4069-a218-22775eedb5e1 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:53.496579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ed156-fb96-4069-a218-22775eedb5e1 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:53.496853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ed156-fb96-4069-a218-22775eedb5e1 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:53.498185 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 528/1088] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:25:53.503113 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d82380-5e82-419c-8b62-a25e46d310a5 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:53.503389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d82380-5e82-419c-8b62-a25e46d310a5 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:53.503657 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d82380-5e82-419c-8b62-a25e46d310a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:53.507199 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789960f6-f7dd-49d1-ae0a-72d6179928d8 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:53.522611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-d3d82380-5e82-419c-8b62-a25e46d310a5 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:53.523816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d3d82380-5e82-419c-8b62-a25e46d310a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:53.542447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a13b8-1300-470e-828d-cd24f0a380ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:53.543045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a13b8-1300-470e-828d-cd24f0a380ed None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:53.543325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a13b8-1300-470e-828d-cd24f0a380ed None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:53.543593 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a13b8-1300-470e-828d-cd24f0a380ed None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:53.544277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a13b8-1300-470e-828d-cd24f0a380ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:53.544574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a13b8-1300-470e-828d-cd24f0a380ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:53.545935 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a13b8-1300-470e-828d-cd24f0a380ed None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:53.554337 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 561/1089] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:25:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:53.560438 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a13b8-1300-470e-828d-cd24f0a380ed None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:53.594970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 529/1090] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:53.619862 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:53.663602 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a13b8-1300-470e-828d-cd24f0a380ed None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:53.699482 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ea07f-a24f-4324-8179-4a7a6349b1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:53.700112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ea07f-a24f-4324-8179-4a7a6349b1eb None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:53.700404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ea07f-a24f-4324-8179-4a7a6349b1eb None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:53.700669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ea07f-a24f-4324-8179-4a7a6349b1eb None admin] PATH_INFO: `/v3/users/ce26a12c29d54d948ad17cb664229971` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:53.701801 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ea07f-a24f-4324-8179-4a7a6349b1eb None admin] RBAC: Authorizing `identity:get_user(user_id=ce26a12c29d54d948ad17cb664229971)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:53.703224 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ea07f-a24f-4324-8179-4a7a6349b1eb None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:53.705250 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 562/1091] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:25:53 2022] GET /identity/v3/users/ce26a12c29d54d948ad17cb664229971 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:53.711827 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d82380-5e82-419c-8b62-a25e46d310a5 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:53.715707 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:53.814586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d82380-5e82-419c-8b62-a25e46d310a5 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:53.850112 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea321f8-f60f-4fb4-a4a6-630f3987e037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:53.850777 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea321f8-f60f-4fb4-a4a6-630f3987e037 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:53.850973 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea321f8-f60f-4fb4-a4a6-630f3987e037 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:53.851238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea321f8-f60f-4fb4-a4a6-630f3987e037 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:53.872433 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ea07f-a24f-4324-8179-4a7a6349b1eb None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:53.925620 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea321f8-f60f-4fb4-a4a6-630f3987e037 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:53.928000 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea321f8-f60f-4fb4-a4a6-630f3987e037 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:53.930873 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:53.978469 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ea07f-a24f-4324-8179-4a7a6349b1eb None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:54.016387 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:54.030542 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c9904-28a9-4ba4-a3a1-8eac44209d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:54.030542 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c9904-28a9-4ba4-a3a1-8eac44209d78 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:54.030542 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c9904-28a9-4ba4-a3a1-8eac44209d78 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:54.030542 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c9904-28a9-4ba4-a3a1-8eac44209d78 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:54.042697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c9904-28a9-4ba4-a3a1-8eac44209d78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:25:54.048753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-681c9904-28a9-4ba4-a3a1-8eac44209d78 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:25:54.048753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:54.048753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:54.048753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:54.048753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:54.048753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:54.048753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:54.048753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:54.048753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:54.048753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:54.048753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:54.048753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:54.048753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:25:54.048753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:25:54.048753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:25:54.048753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:25:54.050679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:25:54.050679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:25:54.050679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:25:54.050679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:25:54.050679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:54.050679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:54.050679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:54.050679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:54.050679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:54.050679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:54.050679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:54.050679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:54.050679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:54.050679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:54.050679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:54.050679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:54.058411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:54.058411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:54.058411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:54.058411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:54.058411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:25:54.058411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:25:54.058411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:25:54.058411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:25:54.058411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:25:54.058411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:25:54.058411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 22:25:54.058411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:54.058411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 563/1092] 10.222.0.59 () {56 vars in 1120 bytes} [Mon Aug 29 22:25:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:25:54.059715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 530/1093] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:25:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:25:54.059792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c9904-28a9-4ba4-a3a1-8eac44209d78 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:54.060003 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea321f8-f60f-4fb4-a4a6-630f3987e037 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:54.181033 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c9904-28a9-4ba4-a3a1-8eac44209d78 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:54.181906 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea321f8-f60f-4fb4-a4a6-630f3987e037 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:54.184457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:54.224939 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da033a35-2210-4878-958b-66e5a36ff5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:54.225603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da033a35-2210-4878-958b-66e5a36ff5b3 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:54.225841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da033a35-2210-4878-958b-66e5a36ff5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:54.226108 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da033a35-2210-4878-958b-66e5a36ff5b3 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:54.226763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da033a35-2210-4878-958b-66e5a36ff5b3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:54.227058 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da033a35-2210-4878-958b-66e5a36ff5b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:54.228484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da033a35-2210-4878-958b-66e5a36ff5b3 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:54.229495 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a8d79-459f-405e-a267-89efefa46167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:54.230135 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a8d79-459f-405e-a267-89efefa46167 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:54.230404 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a8d79-459f-405e-a267-89efefa46167 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:54.230779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a8d79-459f-405e-a267-89efefa46167 None admin] PATH_INFO: `/v3/projects/2ba6be0a6c3e47bcafd8d8be0d24ea1e` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:54.231908 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a8d79-459f-405e-a267-89efefa46167 None admin] RBAC: Authorizing `identity:get_project(project_id=2ba6be0a6c3e47bcafd8d8be0d24ea1e)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:54.233359 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a8d79-459f-405e-a267-89efefa46167 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:54.235074 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 531/1094] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:25:54 2022] GET /identity/v3/projects/2ba6be0a6c3e47bcafd8d8be0d24ea1e => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:54.237309 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 564/1095] 10.222.0.59 () {56 vars in 1115 bytes} [Mon Aug 29 22:25:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:54.241365 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a8d79-459f-405e-a267-89efefa46167 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:54.243563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da033a35-2210-4878-958b-66e5a36ff5b3 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:54.294651 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:54.350290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a8d79-459f-405e-a267-89efefa46167 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:54.352527 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da033a35-2210-4878-958b-66e5a36ff5b3 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:54.389951 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa1a00-4ca1-4bdd-a986-e04ce6d5f4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:54.390622 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa1a00-4ca1-4bdd-a986-e04ce6d5f4a1 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:54.390914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa1a00-4ca1-4bdd-a986-e04ce6d5f4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:54.391187 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa1a00-4ca1-4bdd-a986-e04ce6d5f4a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:54.391931 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20725921-6b67-4abc-a449-847392592f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:54.392618 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20725921-6b67-4abc-a449-847392592f9a None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:54.392887 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20725921-6b67-4abc-a449-847392592f9a None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:54.393195 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20725921-6b67-4abc-a449-847392592f9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:54.393925 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20725921-6b67-4abc-a449-847392592f9a None admin] RBAC: Adding query filter params (role.id=a836108447dc439e994611bc74d53d9c, scope.project.id=2ba6be0a6c3e47bcafd8d8be0d24ea1e, user.id=ce26a12c29d54d948ad17cb664229971) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:54.394221 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20725921-6b67-4abc-a449-847392592f9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:54.396736 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20725921-6b67-4abc-a449-847392592f9a None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:54.403378 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa1a00-4ca1-4bdd-a986-e04ce6d5f4a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:54.404349 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 532/1096] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:25:54 2022] GET /identity/v3/role_assignments?user.id=ce26a12c29d54d948ad17cb664229971&scope.project.id=2ba6be0a6c3e47bcafd8d8be0d24ea1e&role.id=a836108447dc439e994611bc74d53d9c => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:54.404849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa1a00-4ca1-4bdd-a986-e04ce6d5f4a1 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:54.424976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-41aa1a00-4ca1-4bdd-a986-e04ce6d5f4a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:54.424976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:54.424976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:54.424976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:54.424976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:54.424976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:54.424976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:54.424976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:54.424976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:54.424976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:54.424976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:54.424976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:54.424976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:54.424976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:54.424976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:54.424976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:54.426380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:54.426380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:54.426380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:54.426380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:54.426380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:54.426380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:54.426380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:54.426380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:54.426380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:54.426380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:54.426380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:54.426380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:54.426380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:54.426380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:54.426380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:54.426380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:54.426380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:54.427423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:54.427423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:54.427423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:54.427423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 565/1097] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:54.433102 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20725921-6b67-4abc-a449-847392592f9a None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:54.474868 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:54.546980 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20725921-6b67-4abc-a449-847392592f9a None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:54.584078 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26719b-9698-437b-9576-85f8018743e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:54.584723 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26719b-9698-437b-9576-85f8018743e5 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:54.585000 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26719b-9698-437b-9576-85f8018743e5 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:54.585269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26719b-9698-437b-9576-85f8018743e5 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:54.585922 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e26719b-9698-437b-9576-85f8018743e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:54.586244 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e26719b-9698-437b-9576-85f8018743e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:54.587627 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e26719b-9698-437b-9576-85f8018743e5 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:54.600921 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 533/1098] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:54.606727 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa1a00-4ca1-4bdd-a986-e04ce6d5f4a1 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:54.709316 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:54.712380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa1a00-4ca1-4bdd-a986-e04ce6d5f4a1 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:54.747096 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c73ff-9620-4214-9bf2-7d4fe23487e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:54.747725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c73ff-9620-4214-9bf2-7d4fe23487e2 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:54.747995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c73ff-9620-4214-9bf2-7d4fe23487e2 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:54.748304 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c73ff-9620-4214-9bf2-7d4fe23487e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:54.756253 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c73ff-9620-4214-9bf2-7d4fe23487e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:54.757729 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c73ff-9620-4214-9bf2-7d4fe23487e2 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:54.769020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-c44c73ff-9620-4214-9bf2-7d4fe23487e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:25:54.769020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:54.769020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:54.769020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:54.769020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:54.769020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:54.769020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:54.769020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:54.769020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:54.769020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:54.769020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:54.769020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:54.769020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:25:54.769020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:25:54.769020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:25:54.769020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:25:54.770508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:54.770508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:54.770508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:25:54.770508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:54.770508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:25:54.770508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:25:54.770508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:54.770508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:54.770508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:54.770508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:54.770508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:54.770508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:54.770508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:54.770508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:54.770508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:54.770508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:54.770508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:54.770508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:54.771468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:54.771468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:54.771468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:25:54.771468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:25:54.771468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:25:54.771468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:25:54.771468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:25:54.771468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:25:54.771468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 22:25:54.771468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:25:54.771468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 566/1099] 10.222.0.59 () {56 vars in 1093 bytes} [Mon Aug 29 22:25:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:54.776787 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26719b-9698-437b-9576-85f8018743e5 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:54.875459 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:54.881869 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26719b-9698-437b-9576-85f8018743e5 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:54.919018 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2a131-01a3-4b54-8b98-531631c94856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:54.919018 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2a131-01a3-4b54-8b98-531631c94856 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:54.919018 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2a131-01a3-4b54-8b98-531631c94856 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:54.919018 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2a131-01a3-4b54-8b98-531631c94856 None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:54.919981 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b2a131-01a3-4b54-8b98-531631c94856 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:54.920249 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b2a131-01a3-4b54-8b98-531631c94856 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:54.921938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b2a131-01a3-4b54-8b98-531631c94856 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:54.940551 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 534/1100] 10.222.0.59 () {56 vars in 1133 bytes} [Mon Aug 29 22:25:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:54.947004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c73ff-9620-4214-9bf2-7d4fe23487e2 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:55.049691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c73ff-9620-4214-9bf2-7d4fe23487e2 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:55.084777 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e6026-0c1f-401b-967b-1c1a96c2c53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:55.085375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e6026-0c1f-401b-967b-1c1a96c2c53c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:55.085650 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e6026-0c1f-401b-967b-1c1a96c2c53c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:55.085918 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e6026-0c1f-401b-967b-1c1a96c2c53c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:55.116315 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:55.156958 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e6026-0c1f-401b-967b-1c1a96c2c53c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:55.158396 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e6026-0c1f-401b-967b-1c1a96c2c53c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:55.258717 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 567/1101] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:25:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:25:55.264743 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2a131-01a3-4b54-8b98-531631c94856 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:55.367248 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2a131-01a3-4b54-8b98-531631c94856 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:55.402275 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe74ce-0dcf-4a26-b19c-aad0de0a9e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:55.402902 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe74ce-0dcf-4a26-b19c-aad0de0a9e19 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:55.403174 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe74ce-0dcf-4a26-b19c-aad0de0a9e19 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:55.403442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe74ce-0dcf-4a26-b19c-aad0de0a9e19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:55.414127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe74ce-0dcf-4a26-b19c-aad0de0a9e19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:55.415563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe74ce-0dcf-4a26-b19c-aad0de0a9e19 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:55.431116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-c1fe74ce-0dcf-4a26-b19c-aad0de0a9e19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:55.431116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:55.431116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:55.431116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:55.431116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:55.431116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:55.431116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:55.431116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:55.431116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:55.431116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:55.431116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:55.431116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:55.431116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:25:55.431116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:25:55.431116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:25:55.431116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:25:55.432240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:55.432240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:55.432240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:55.432240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:55.432240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:55.432240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:55.432240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:55.432240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:55.432240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:55.432240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:55.432240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:55.432240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:55.432240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:55.432240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:55.432240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:55.432240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:55.432240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:25:55.433167 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:25:55.433167 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:25:55.433167 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:55.433167 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 535/1102] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:55.438971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e6026-0c1f-401b-967b-1c1a96c2c53c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:55.501266 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:55.544853 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e6026-0c1f-401b-967b-1c1a96c2c53c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:55.580891 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c29148-f40a-4853-8a62-a3d5e43ef57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:55.581488 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c29148-f40a-4853-8a62-a3d5e43ef57f None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:55.581761 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c29148-f40a-4853-8a62-a3d5e43ef57f None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:55.582565 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c29148-f40a-4853-8a62-a3d5e43ef57f None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:55.583273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c29148-f40a-4853-8a62-a3d5e43ef57f None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:55.583566 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c29148-f40a-4853-8a62-a3d5e43ef57f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:55.585033 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c29148-f40a-4853-8a62-a3d5e43ef57f None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:55.596344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 568/1103] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:55.603108 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe74ce-0dcf-4a26-b19c-aad0de0a9e19 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:55.612946 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:55.705584 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe74ce-0dcf-4a26-b19c-aad0de0a9e19 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:55.741208 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fbc1d-503f-481b-8e27-838829096da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:55.741832 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fbc1d-503f-481b-8e27-838829096da7 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:55.742105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fbc1d-503f-481b-8e27-838829096da7 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:55.742370 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fbc1d-503f-481b-8e27-838829096da7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:55.749040 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1fbc1d-503f-481b-8e27-838829096da7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:55.750466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1fbc1d-503f-481b-8e27-838829096da7 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:55.762453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-0d1fbc1d-503f-481b-8e27-838829096da7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:55.762453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:25:55.762453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:25:55.762453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:25:55.762453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:25:55.762453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:25:55.762453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:25:55.762453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:25:55.762453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:25:55.762453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:25:55.762453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:25:55.762453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:25:55.762453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:25:55.762453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:25:55.762453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:25:55.763603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:25:55.763603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:25:55.763603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:25:55.763603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:25:55.763603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:25:55.763603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:25:55.763603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:25:55.763603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:25:55.763603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:25:55.763603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:25:55.763603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:25:55.763603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:25:55.763603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:25:55.763603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:25:55.763603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:25:55.763603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:25:55.763603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:25:55.764561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:25:55.764561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:25:55.764561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:25:55.764561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:25:55.764561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:25:55.764561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:25:55.764561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:25:55.764561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:25:55.764561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 536/1104] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:25:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:25:55.770394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c29148-f40a-4853-8a62-a3d5e43ef57f None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:55.872978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c29148-f40a-4853-8a62-a3d5e43ef57f None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:55.892709 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:55.907184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176fd5d7-4587-4e89-9e20-e9e8aae62f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:55.907796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fd5d7-4587-4e89-9e20-e9e8aae62f14 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:55.908073 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fd5d7-4587-4e89-9e20-e9e8aae62f14 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:55.908373 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fd5d7-4587-4e89-9e20-e9e8aae62f14 None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:55.909064 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176fd5d7-4587-4e89-9e20-e9e8aae62f14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:55.909369 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176fd5d7-4587-4e89-9e20-e9e8aae62f14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:55.911439 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176fd5d7-4587-4e89-9e20-e9e8aae62f14 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:55.924193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 569/1105] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:25:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:55.930996 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fbc1d-503f-481b-8e27-838829096da7 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:55.998894 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:56.033463 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fbc1d-503f-481b-8e27-838829096da7 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:56.068510 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b729dd-ca39-4087-bc25-197ae4d12414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:56.069123 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b729dd-ca39-4087-bc25-197ae4d12414 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:56.069406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b729dd-ca39-4087-bc25-197ae4d12414 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:56.069676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b729dd-ca39-4087-bc25-197ae4d12414 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:56.070415 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b729dd-ca39-4087-bc25-197ae4d12414 None admin] RBAC: Adding query filter params (role.id=04230301829e46deb6a9a0f33b868e35, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=7f21105679f245a19ea4850e6c010dbf) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:25:56.070744 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b729dd-ca39-4087-bc25-197ae4d12414 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:56.072177 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b729dd-ca39-4087-bc25-197ae4d12414 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:56.078353 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 537/1106] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:25:55 2022] GET /identity/v3/role_assignments?user.id=7f21105679f245a19ea4850e6c010dbf&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=04230301829e46deb6a9a0f33b868e35 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:25:56.397599 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d526c4-f64e-4596-aa86-73cf89c87d6a None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:56.398115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d526c4-f64e-4596-aa86-73cf89c87d6a None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:56.398248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d526c4-f64e-4596-aa86-73cf89c87d6a None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:56.399611 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 570/1107] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:56.405476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69b75e-33f9-495d-99bc-2fe72f2a4ac5 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:56.405760 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69b75e-33f9-495d-99bc-2fe72f2a4ac5 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:56.406034 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69b75e-33f9-495d-99bc-2fe72f2a4ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:56.425992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-0c69b75e-33f9-495d-99bc-2fe72f2a4ac5 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:56.427288 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-0c69b75e-33f9-495d-99bc-2fe72f2a4ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:56.503103 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 538/1108] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:25:56.774283 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d526c4-f64e-4596-aa86-73cf89c87d6a None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:56.893486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d526c4-f64e-4596-aa86-73cf89c87d6a None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:56.936417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab15c43-49b5-4558-94f1-7ba54204ce64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:56.937050 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab15c43-49b5-4558-94f1-7ba54204ce64 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:56.937325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab15c43-49b5-4558-94f1-7ba54204ce64 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:56.937592 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab15c43-49b5-4558-94f1-7ba54204ce64 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:56.938727 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab15c43-49b5-4558-94f1-7ba54204ce64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:56.940176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab15c43-49b5-4558-94f1-7ba54204ce64 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:56.942546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 571/1109] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:25:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:56.948813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69b75e-33f9-495d-99bc-2fe72f2a4ac5 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:57.016453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:57.058796 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69b75e-33f9-495d-99bc-2fe72f2a4ac5 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:57.095045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de175861-51de-4b40-8635-15ceb8097ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:57.095647 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de175861-51de-4b40-8635-15ceb8097ab4 None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:57.095919 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de175861-51de-4b40-8635-15ceb8097ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:57.096205 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de175861-51de-4b40-8635-15ceb8097ab4 None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:57.096849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de175861-51de-4b40-8635-15ceb8097ab4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:57.098413 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de175861-51de-4b40-8635-15ceb8097ab4 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:57.203550 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 539/1110] 10.222.0.59 () {60 vars in 1132 bytes} [Mon Aug 29 22:25:56 2022] POST /identity/v3/users => generated 286 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:25:59.185036 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517255a3-3268-4db6-9090-0bdc7a6d69fd None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:59.185535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517255a3-3268-4db6-9090-0bdc7a6d69fd None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:59.185535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517255a3-3268-4db6-9090-0bdc7a6d69fd None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:59.186864 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 572/1111] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:25:59.192750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593b476-20c9-4864-8ae4-f7ea7f91759c None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:59.193241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593b476-20c9-4864-8ae4-f7ea7f91759c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:59.193241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593b476-20c9-4864-8ae4-f7ea7f91759c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:59.212389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-b593b476-20c9-4864-8ae4-f7ea7f91759c None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:25:59.213642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b593b476-20c9-4864-8ae4-f7ea7f91759c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:25:59.284734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 540/1112] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:25:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:25:59.554248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517255a3-3268-4db6-9090-0bdc7a6d69fd None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:59.611319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:59.663670 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517255a3-3268-4db6-9090-0bdc7a6d69fd None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:59.698022 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418249d6-3c64-48ef-ae23-9fced7407fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:59.698643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418249d6-3c64-48ef-ae23-9fced7407fc8 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:59.698977 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418249d6-3c64-48ef-ae23-9fced7407fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:59.699247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418249d6-3c64-48ef-ae23-9fced7407fc8 None admin] PATH_INFO: `/v3/roles/733cd39702ab4cbc860e34886dbb165f` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:59.700680 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418249d6-3c64-48ef-ae23-9fced7407fc8 None admin] RBAC: Authorizing `identity:get_role(role_id=733cd39702ab4cbc860e34886dbb165f)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:59.702042 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418249d6-3c64-48ef-ae23-9fced7407fc8 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:59.703593 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 573/1113] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:25:59 2022] GET /identity/v3/roles/733cd39702ab4cbc860e34886dbb165f => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:59.710480 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593b476-20c9-4864-8ae4-f7ea7f91759c None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:59.759673 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:25:59.815135 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593b476-20c9-4864-8ae4-f7ea7f91759c None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:25:59.853555 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10719ef0-8858-4fdb-9ec3-fcc0f3c275bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:25:59.854184 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10719ef0-8858-4fdb-9ec3-fcc0f3c275bc None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:25:59.854465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10719ef0-8858-4fdb-9ec3-fcc0f3c275bc None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:25:59.854738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10719ef0-8858-4fdb-9ec3-fcc0f3c275bc None admin] PATH_INFO: `/v3/users/ccc1d1c0d00c436ebaf18df0fc0b025d` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:25:59.870828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10719ef0-8858-4fdb-9ec3-fcc0f3c275bc None admin] RBAC: Authorizing `identity:get_user(user_id=ccc1d1c0d00c436ebaf18df0fc0b025d)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:25:59.872304 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10719ef0-8858-4fdb-9ec3-fcc0f3c275bc None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:25:59.874041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 541/1114] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:25:59 2022] GET /identity/v3/users/ccc1d1c0d00c436ebaf18df0fc0b025d => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:25:59.879898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418249d6-3c64-48ef-ae23-9fced7407fc8 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:25:59.987605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418249d6-3c64-48ef-ae23-9fced7407fc8 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:00.009001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:00.028151 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19af66-4d7e-4d50-8088-efb5e914cf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:00.028766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19af66-4d7e-4d50-8088-efb5e914cf88 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:00.029043 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19af66-4d7e-4d50-8088-efb5e914cf88 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:00.029310 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19af66-4d7e-4d50-8088-efb5e914cf88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:00.102829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19af66-4d7e-4d50-8088-efb5e914cf88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:00.104210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19af66-4d7e-4d50-8088-efb5e914cf88 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:00.208174 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 574/1115] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:25:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:26:00.214401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10719ef0-8858-4fdb-9ec3-fcc0f3c275bc None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:00.319884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10719ef0-8858-4fdb-9ec3-fcc0f3c275bc None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:00.322380 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:00.355071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3976b-e883-4fb2-b7fa-d9b831a48ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:00.355613 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3976b-e883-4fb2-b7fa-d9b831a48ea4 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:00.355913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3976b-e883-4fb2-b7fa-d9b831a48ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:00.356262 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3976b-e883-4fb2-b7fa-d9b831a48ea4 None admin] PATH_INFO: `/v3/projects/2ba6be0a6c3e47bcafd8d8be0d24ea1e` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:00.357327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae3976b-e883-4fb2-b7fa-d9b831a48ea4 None admin] RBAC: Authorizing `identity:get_project(project_id=2ba6be0a6c3e47bcafd8d8be0d24ea1e)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:00.358738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae3976b-e883-4fb2-b7fa-d9b831a48ea4 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:00.360391 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 542/1116] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:26:00 2022] GET /identity/v3/projects/2ba6be0a6c3e47bcafd8d8be0d24ea1e => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:00.367552 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19af66-4d7e-4d50-8088-efb5e914cf88 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:00.385288 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:00.470477 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19af66-4d7e-4d50-8088-efb5e914cf88 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:00.507780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca624d7-dcf9-4244-8048-8de754c9f0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:00.508405 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca624d7-dcf9-4244-8048-8de754c9f0ad None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:00.508681 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca624d7-dcf9-4244-8048-8de754c9f0ad None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:00.508946 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca624d7-dcf9-4244-8048-8de754c9f0ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:00.509670 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca624d7-dcf9-4244-8048-8de754c9f0ad None admin] RBAC: Adding query filter params (role.id=733cd39702ab4cbc860e34886dbb165f, scope.project.id=2ba6be0a6c3e47bcafd8d8be0d24ea1e, user.id=ccc1d1c0d00c436ebaf18df0fc0b025d) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:26:00.509972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca624d7-dcf9-4244-8048-8de754c9f0ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:00.511391 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca624d7-dcf9-4244-8048-8de754c9f0ad None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:00.517130 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 575/1117] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:26:00 2022] GET /identity/v3/role_assignments?user.id=ccc1d1c0d00c436ebaf18df0fc0b025d&scope.project.id=2ba6be0a6c3e47bcafd8d8be0d24ea1e&role.id=733cd39702ab4cbc860e34886dbb165f => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:02.424432 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31d7f6-5c15-4f1d-bd85-330eb59c4f5f None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:02.425060 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31d7f6-5c15-4f1d-bd85-330eb59c4f5f None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:02.425060 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31d7f6-5c15-4f1d-bd85-330eb59c4f5f None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:02.426342 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 543/1118] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:26:02.431784 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931fd6c-f0b5-4e64-9e21-8929061e906b None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:02.432303 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931fd6c-f0b5-4e64-9e21-8929061e906b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:02.432419 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931fd6c-f0b5-4e64-9e21-8929061e906b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:02.450563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-a931fd6c-f0b5-4e64-9e21-8929061e906b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:26:02.451758 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a931fd6c-f0b5-4e64-9e21-8929061e906b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:26:02.524686 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 576/1119] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:26:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:26:02.794772 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31d7f6-5c15-4f1d-bd85-330eb59c4f5f None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:02.899749 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31d7f6-5c15-4f1d-bd85-330eb59c4f5f None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:02.933873 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c29e9-3420-4d88-ac70-bdde9fe3fab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:02.934502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c29e9-3420-4d88-ac70-bdde9fe3fab2 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:02.934778 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c29e9-3420-4d88-ac70-bdde9fe3fab2 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:02.935045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c29e9-3420-4d88-ac70-bdde9fe3fab2 None admin] PATH_INFO: `/v3/roles/733cd39702ab4cbc860e34886dbb165f` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:02.936500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c29e9-3420-4d88-ac70-bdde9fe3fab2 None admin] RBAC: Authorizing `identity:get_role(role_id=733cd39702ab4cbc860e34886dbb165f)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:02.937849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c29e9-3420-4d88-ac70-bdde9fe3fab2 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:02.939184 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 544/1120] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:26:02 2022] GET /identity/v3/roles/733cd39702ab4cbc860e34886dbb165f => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:02.951311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931fd6c-f0b5-4e64-9e21-8929061e906b None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:02.965018 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:03.062976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931fd6c-f0b5-4e64-9e21-8929061e906b None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:03.098564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eefec7-81ed-4364-a6b0-cb660787e441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:03.098564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eefec7-81ed-4364-a6b0-cb660787e441 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:03.099582 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eefec7-81ed-4364-a6b0-cb660787e441 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:03.099778 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eefec7-81ed-4364-a6b0-cb660787e441 None admin] PATH_INFO: `/v3/users/ccc1d1c0d00c436ebaf18df0fc0b025d` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:03.101086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eefec7-81ed-4364-a6b0-cb660787e441 None admin] RBAC: Authorizing `identity:get_user(user_id=ccc1d1c0d00c436ebaf18df0fc0b025d)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:03.102420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eefec7-81ed-4364-a6b0-cb660787e441 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:03.104021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 577/1121] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:26:02 2022] GET /identity/v3/users/ccc1d1c0d00c436ebaf18df0fc0b025d => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:03.109932 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c29e9-3420-4d88-ac70-bdde9fe3fab2 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:03.153729 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:03.213957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c29e9-3420-4d88-ac70-bdde9fe3fab2 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:03.250246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041486a-6016-4ce5-9d39-f96de3b74d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:03.250872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041486a-6016-4ce5-9d39-f96de3b74d93 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:03.251149 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041486a-6016-4ce5-9d39-f96de3b74d93 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:03.251418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041486a-6016-4ce5-9d39-f96de3b74d93 None admin] PATH_INFO: `/v3/projects/2ba6be0a6c3e47bcafd8d8be0d24ea1e` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:03.252564 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041486a-6016-4ce5-9d39-f96de3b74d93 None admin] RBAC: Authorizing `identity:get_project(project_id=2ba6be0a6c3e47bcafd8d8be0d24ea1e)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:03.253996 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041486a-6016-4ce5-9d39-f96de3b74d93 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:03.256313 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 545/1122] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:26:03 2022] GET /identity/v3/projects/2ba6be0a6c3e47bcafd8d8be0d24ea1e => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:03.262716 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eefec7-81ed-4364-a6b0-cb660787e441 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:03.348838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:03.370464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eefec7-81ed-4364-a6b0-cb660787e441 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:03.406004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4a59f-5a51-4ee6-acad-6e153eed550f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:03.406656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4a59f-5a51-4ee6-acad-6e153eed550f None admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:03.406931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4a59f-5a51-4ee6-acad-6e153eed550f None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:03.407207 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4a59f-5a51-4ee6-acad-6e153eed550f None admin] PATH_INFO: `/v3/projects/2ba6be0a6c3e47bcafd8d8be0d24ea1e/users/ccc1d1c0d00c436ebaf18df0fc0b025d/roles/733cd39702ab4cbc860e34886dbb165f` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:03.409400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc4a59f-5a51-4ee6-acad-6e153eed550f None admin] RBAC: Authorizing `identity:create_grant(project_id=2ba6be0a6c3e47bcafd8d8be0d24ea1e, user_id=ccc1d1c0d00c436ebaf18df0fc0b025d, role_id=733cd39702ab4cbc860e34886dbb165f)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:03.410822 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc4a59f-5a51-4ee6-acad-6e153eed550f None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:03.496517 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 578/1123] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:26:03 2022] PUT /identity/v3/projects/2ba6be0a6c3e47bcafd8d8be0d24ea1e/users/ccc1d1c0d00c436ebaf18df0fc0b025d/roles/733cd39702ab4cbc860e34886dbb165f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:26:05.427828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c52d4-a16d-42bb-b4cf-88ef0f489a24 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:05.428387 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c52d4-a16d-42bb-b4cf-88ef0f489a24 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:05.428387 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c52d4-a16d-42bb-b4cf-88ef0f489a24 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:05.429717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 546/1124] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:26:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:26:05.435009 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8cab8-6c57-43ce-8a11-fb2ae33e021b None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:05.438522 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8cab8-6c57-43ce-8a11-fb2ae33e021b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:05.438522 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8cab8-6c57-43ce-8a11-fb2ae33e021b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:05.454468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-67a8cab8-6c57-43ce-8a11-fb2ae33e021b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:26:05.455652 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-67a8cab8-6c57-43ce-8a11-fb2ae33e021b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:26:05.530075 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 579/1125] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:26:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:26:05.804300 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c52d4-a16d-42bb-b4cf-88ef0f489a24 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:05.905692 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c52d4-a16d-42bb-b4cf-88ef0f489a24 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:05.916232 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:05.941310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee4c9a-0335-4542-b848-71646cabe482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:05.941930 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee4c9a-0335-4542-b848-71646cabe482 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:05.942824 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee4c9a-0335-4542-b848-71646cabe482 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:05.943124 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee4c9a-0335-4542-b848-71646cabe482 None admin] PATH_INFO: `/v3/roles/733cd39702ab4cbc860e34886dbb165f` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:05.944673 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee4c9a-0335-4542-b848-71646cabe482 None admin] RBAC: Authorizing `identity:get_role(role_id=733cd39702ab4cbc860e34886dbb165f)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:05.946570 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee4c9a-0335-4542-b848-71646cabe482 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:05.948199 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 547/1126] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:26:05 2022] GET /identity/v3/roles/733cd39702ab4cbc860e34886dbb165f => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:05.954177 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8cab8-6c57-43ce-8a11-fb2ae33e021b None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:06.018471 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:06.050750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8cab8-6c57-43ce-8a11-fb2ae33e021b None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:06.086238 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5256350-4450-43df-9611-f45809462a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:06.086767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5256350-4450-43df-9611-f45809462a84 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:06.087039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5256350-4450-43df-9611-f45809462a84 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:06.087301 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5256350-4450-43df-9611-f45809462a84 None admin] PATH_INFO: `/v3/users/ccc1d1c0d00c436ebaf18df0fc0b025d` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:06.088416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5256350-4450-43df-9611-f45809462a84 None admin] RBAC: Authorizing `identity:get_user(user_id=ccc1d1c0d00c436ebaf18df0fc0b025d)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:06.089946 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5256350-4450-43df-9611-f45809462a84 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:06.091514 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 580/1127] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:26:05 2022] GET /identity/v3/users/ccc1d1c0d00c436ebaf18df0fc0b025d => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:06.097847 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee4c9a-0335-4542-b848-71646cabe482 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:06.195803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee4c9a-0335-4542-b848-71646cabe482 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:06.229456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d056cc-d92d-48d7-b30b-010bfb30008d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:06.230046 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d056cc-d92d-48d7-b30b-010bfb30008d None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:06.230316 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d056cc-d92d-48d7-b30b-010bfb30008d None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:06.230581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d056cc-d92d-48d7-b30b-010bfb30008d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:06.275911 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:06.309809 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d056cc-d92d-48d7-b30b-010bfb30008d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:06.313070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d056cc-d92d-48d7-b30b-010bfb30008d None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:06.415561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 548/1128] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:26:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:26:06.421868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5256350-4450-43df-9611-f45809462a84 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:06.520403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5256350-4450-43df-9611-f45809462a84 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:06.554436 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa1577-919b-49ca-a2f4-481683d7f58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:06.555049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa1577-919b-49ca-a2f4-481683d7f58f None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:06.555322 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa1577-919b-49ca-a2f4-481683d7f58f None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:06.555586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa1577-919b-49ca-a2f4-481683d7f58f None admin] PATH_INFO: `/v3/projects/2ba6be0a6c3e47bcafd8d8be0d24ea1e` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:06.556700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa1577-919b-49ca-a2f4-481683d7f58f None admin] RBAC: Authorizing `identity:get_project(project_id=2ba6be0a6c3e47bcafd8d8be0d24ea1e)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:06.558116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa1577-919b-49ca-a2f4-481683d7f58f None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:06.559728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 581/1129] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:26:06 2022] GET /identity/v3/projects/2ba6be0a6c3e47bcafd8d8be0d24ea1e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:06.566237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d056cc-d92d-48d7-b30b-010bfb30008d None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:06.643431 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:06.666167 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d056cc-d92d-48d7-b30b-010bfb30008d None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:06.700728 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a46dbd0-db73-4655-8512-a07d0e32e33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:06.701323 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a46dbd0-db73-4655-8512-a07d0e32e33b None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:06.701598 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a46dbd0-db73-4655-8512-a07d0e32e33b None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:06.701864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a46dbd0-db73-4655-8512-a07d0e32e33b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:06.702581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a46dbd0-db73-4655-8512-a07d0e32e33b None admin] RBAC: Adding query filter params (role.id=733cd39702ab4cbc860e34886dbb165f, scope.project.id=2ba6be0a6c3e47bcafd8d8be0d24ea1e, user.id=ccc1d1c0d00c436ebaf18df0fc0b025d) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:26:06.702878 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a46dbd0-db73-4655-8512-a07d0e32e33b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:06.704329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a46dbd0-db73-4655-8512-a07d0e32e33b None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:06.710442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 549/1130] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:26:06 2022] GET /identity/v3/role_assignments?user.id=ccc1d1c0d00c436ebaf18df0fc0b025d&scope.project.id=2ba6be0a6c3e47bcafd8d8be0d24ea1e&role.id=733cd39702ab4cbc860e34886dbb165f => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:08.703506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31040595-f0be-4544-b342-a4cd596a06a8 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:08.704039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31040595-f0be-4544-b342-a4cd596a06a8 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:08.704039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31040595-f0be-4544-b342-a4cd596a06a8 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:08.705404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 582/1131] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:26:08.711255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cabfb-71ea-4bd5-90cd-df8605dfd8a1 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:08.711676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cabfb-71ea-4bd5-90cd-df8605dfd8a1 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:08.711832 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cabfb-71ea-4bd5-90cd-df8605dfd8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:08.731220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-206cabfb-71ea-4bd5-90cd-df8605dfd8a1 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:26:08.732443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-206cabfb-71ea-4bd5-90cd-df8605dfd8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:26:08.800669 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 550/1132] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:26:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:26:09.074062 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31040595-f0be-4544-b342-a4cd596a06a8 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:09.077928 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:09.181541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31040595-f0be-4544-b342-a4cd596a06a8 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:09.215631 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b43bf-1d60-4857-8131-f6e23493294f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:09.216275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b43bf-1d60-4857-8131-f6e23493294f None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:09.216551 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b43bf-1d60-4857-8131-f6e23493294f None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:09.216818 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b43bf-1d60-4857-8131-f6e23493294f None admin] PATH_INFO: `/v3/domains/default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:09.217914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b43bf-1d60-4857-8131-f6e23493294f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:09.219329 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b43bf-1d60-4857-8131-f6e23493294f None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:09.220938 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 583/1133] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:26:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:09.228173 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206cabfb-71ea-4bd5-90cd-df8605dfd8a1 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:09.298738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:09.330502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206cabfb-71ea-4bd5-90cd-df8605dfd8a1 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:09.365963 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64cc516-47d6-4cf1-9e72-89c46277e6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:09.366564 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64cc516-47d6-4cf1-9e72-89c46277e6e9 None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:09.366840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64cc516-47d6-4cf1-9e72-89c46277e6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:09.367106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64cc516-47d6-4cf1-9e72-89c46277e6e9 None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:09.367756 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64cc516-47d6-4cf1-9e72-89c46277e6e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:09.369216 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64cc516-47d6-4cf1-9e72-89c46277e6e9 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:09.462130 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 551/1134] 10.222.0.59 () {60 vars in 1141 bytes} [Mon Aug 29 22:26:09 2022] POST /identity/v3/projects => generated 311 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:26:11.441085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3afdd3-77bf-4f09-ad4c-f3f4e6acc99b None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:11.441856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3afdd3-77bf-4f09-ad4c-f3f4e6acc99b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:11.441856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3afdd3-77bf-4f09-ad4c-f3f4e6acc99b None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:11.443005 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 584/1135] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:26:11.448766 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7a202-eb4b-4dae-b774-b53d33a11ae4 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:11.449270 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7a202-eb4b-4dae-b774-b53d33a11ae4 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:11.449386 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7a202-eb4b-4dae-b774-b53d33a11ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:11.469443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-52f7a202-eb4b-4dae-b774-b53d33a11ae4 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:26:11.470663 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-52f7a202-eb4b-4dae-b774-b53d33a11ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:26:11.546812 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 552/1136] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:26:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:26:11.821336 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3afdd3-77bf-4f09-ad4c-f3f4e6acc99b None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:11.914991 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:11.953243 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3afdd3-77bf-4f09-ad4c-f3f4e6acc99b None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:11.997271 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4d49c-7aac-42d2-bf77-6e817a041899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:11.997906 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4d49c-7aac-42d2-bf77-6e817a041899 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:11.998179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4d49c-7aac-42d2-bf77-6e817a041899 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:11.998447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4d49c-7aac-42d2-bf77-6e817a041899 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:11.999569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4d49c-7aac-42d2-bf77-6e817a041899 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:12.001030 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4d49c-7aac-42d2-bf77-6e817a041899 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:12.002610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 585/1137] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:26:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:12.009064 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f7a202-eb4b-4dae-b774-b53d33a11ae4 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:12.067806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:12.122912 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f7a202-eb4b-4dae-b774-b53d33a11ae4 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:12.160471 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac929426-d17b-4f45-9af6-3cac4fa0d4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:12.161663 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac929426-d17b-4f45-9af6-3cac4fa0d4b3 None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:12.161933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac929426-d17b-4f45-9af6-3cac4fa0d4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:12.162196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac929426-d17b-4f45-9af6-3cac4fa0d4b3 None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:12.162852 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac929426-d17b-4f45-9af6-3cac4fa0d4b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:12.164354 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac929426-d17b-4f45-9af6-3cac4fa0d4b3 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:12.249230 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 553/1138] 10.222.0.59 () {60 vars in 1132 bytes} [Mon Aug 29 22:26:12 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:26:14.260222 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc186b-8e69-45f8-a45a-e31fb11f4545 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:14.260754 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc186b-8e69-45f8-a45a-e31fb11f4545 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:14.260754 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc186b-8e69-45f8-a45a-e31fb11f4545 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:14.262041 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 586/1139] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:26:14.267549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d7a46-90f6-4df5-beb8-658c6481c6eb None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:14.268015 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d7a46-90f6-4df5-beb8-658c6481c6eb None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:14.268105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d7a46-90f6-4df5-beb8-658c6481c6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:14.287639 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-aa6d7a46-90f6-4df5-beb8-658c6481c6eb None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:26:14.288881 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-aa6d7a46-90f6-4df5-beb8-658c6481c6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:26:14.361603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 554/1140] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:26:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:26:14.634940 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc186b-8e69-45f8-a45a-e31fb11f4545 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:14.743060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc186b-8e69-45f8-a45a-e31fb11f4545 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:14.779564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35716998-4fa9-4d84-85ca-942268c7ff1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:14.780209 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35716998-4fa9-4d84-85ca-942268c7ff1c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:14.780484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35716998-4fa9-4d84-85ca-942268c7ff1c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:14.780753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35716998-4fa9-4d84-85ca-942268c7ff1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:14.789033 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:14.792076 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35716998-4fa9-4d84-85ca-942268c7ff1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:26:14.796933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-35716998-4fa9-4d84-85ca-942268c7ff1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:26:14.796933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:26:14.796933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:26:14.796933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:26:14.796933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:26:14.796933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:26:14.796933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:26:14.796933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:26:14.796933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:26:14.796933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:26:14.796933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:26:14.796933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:26:14.796933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:26:14.796933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:26:14.796933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:26:14.796933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:26:14.802586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:26:14.802586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:26:14.802586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:26:14.802586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:26:14.802586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:26:14.802586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:26:14.802586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:26:14.802586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:26:14.802586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:26:14.802586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:26:14.802586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:26:14.802586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:26:14.802586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:26:14.802586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:26:14.802586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:26:14.802586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:26:14.803715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:26:14.803715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:26:14.803715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:26:14.803715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:26:14.803715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:26:14.803715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:26:14.803715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:26:14.803715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:26:14.803715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:26:14.803715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:26:14.803715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:26:14.803715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:26:14.803715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 587/1141] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:26:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:26:14.804717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d7a46-90f6-4df5-beb8-658c6481c6eb None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:14.852272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:14.920291 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d7a46-90f6-4df5-beb8-658c6481c6eb None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:14.958035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51055c31-8390-4179-8f41-06f38d690a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:14.958766 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51055c31-8390-4179-8f41-06f38d690a3f None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:14.959042 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51055c31-8390-4179-8f41-06f38d690a3f None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:14.959304 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51055c31-8390-4179-8f41-06f38d690a3f None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:14.959962 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51055c31-8390-4179-8f41-06f38d690a3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:26:14.960291 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51055c31-8390-4179-8f41-06f38d690a3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:14.961669 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51055c31-8390-4179-8f41-06f38d690a3f None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:14.970481 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 555/1142] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:26:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:14.976628 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35716998-4fa9-4d84-85ca-942268c7ff1c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:15.085428 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35716998-4fa9-4d84-85ca-942268c7ff1c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:15.125743 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034c316-e23d-4f54-b97f-b867e81e7ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:15.126365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034c316-e23d-4f54-b97f-b867e81e7ad7 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:15.126643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034c316-e23d-4f54-b97f-b867e81e7ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:15.126960 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034c316-e23d-4f54-b97f-b867e81e7ad7 None admin] PATH_INFO: `/v3/users/7b76dbfdaf21434bbd782603ba27f88c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:15.141463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034c316-e23d-4f54-b97f-b867e81e7ad7 None admin] RBAC: Authorizing `identity:get_user(user_id=7b76dbfdaf21434bbd782603ba27f88c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:15.142912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034c316-e23d-4f54-b97f-b867e81e7ad7 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:15.144736 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 588/1143] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:26:14 2022] GET /identity/v3/users/7b76dbfdaf21434bbd782603ba27f88c => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:15.151306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51055c31-8390-4179-8f41-06f38d690a3f None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:15.262610 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:15.265625 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51055c31-8390-4179-8f41-06f38d690a3f None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:15.300266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bef6c6-a2f1-40af-ab2b-4bf1194a4593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:15.300965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bef6c6-a2f1-40af-ab2b-4bf1194a4593 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:15.301125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bef6c6-a2f1-40af-ab2b-4bf1194a4593 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:15.301380 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bef6c6-a2f1-40af-ab2b-4bf1194a4593 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:15.371969 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bef6c6-a2f1-40af-ab2b-4bf1194a4593 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:15.373456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bef6c6-a2f1-40af-ab2b-4bf1194a4593 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:15.477740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:15.483211 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 556/1144] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:26:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:26:15.489892 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034c316-e23d-4f54-b97f-b867e81e7ad7 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:15.550877 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:15.595428 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034c316-e23d-4f54-b97f-b867e81e7ad7 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:15.632123 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314ef59-7c5d-4964-a9c5-f033f7730018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:15.632782 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314ef59-7c5d-4964-a9c5-f033f7730018 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:15.633058 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314ef59-7c5d-4964-a9c5-f033f7730018 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:15.633330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314ef59-7c5d-4964-a9c5-f033f7730018 None admin] PATH_INFO: `/v3/projects/7b3041d26908404f99d8acc5d30f4b4e` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:15.634473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314ef59-7c5d-4964-a9c5-f033f7730018 None admin] RBAC: Authorizing `identity:get_project(project_id=7b3041d26908404f99d8acc5d30f4b4e)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:15.635950 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314ef59-7c5d-4964-a9c5-f033f7730018 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:15.637632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 589/1145] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:26:15 2022] GET /identity/v3/projects/7b3041d26908404f99d8acc5d30f4b4e => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:26:15.643933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bef6c6-a2f1-40af-ab2b-4bf1194a4593 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:15.755587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bef6c6-a2f1-40af-ab2b-4bf1194a4593 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:15.791532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854149f8-d496-4466-91d2-b142061f5bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:15.792168 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854149f8-d496-4466-91d2-b142061f5bd6 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:15.792467 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854149f8-d496-4466-91d2-b142061f5bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:15.792746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854149f8-d496-4466-91d2-b142061f5bd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:15.793480 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854149f8-d496-4466-91d2-b142061f5bd6 None admin] RBAC: Adding query filter params (role.id=a836108447dc439e994611bc74d53d9c, scope.project.id=7b3041d26908404f99d8acc5d30f4b4e, user.id=7b76dbfdaf21434bbd782603ba27f88c) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:26:15.793784 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854149f8-d496-4466-91d2-b142061f5bd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:15.795241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854149f8-d496-4466-91d2-b142061f5bd6 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:15.802306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 557/1146] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:26:15 2022] GET /identity/v3/role_assignments?user.id=7b76dbfdaf21434bbd782603ba27f88c&scope.project.id=7b3041d26908404f99d8acc5d30f4b4e&role.id=a836108447dc439e994611bc74d53d9c => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:17.731059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a1a82-c15e-4571-a0d2-c2d7ec0fba1b None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:17.731702 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a1a82-c15e-4571-a0d2-c2d7ec0fba1b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:17.731702 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a1a82-c15e-4571-a0d2-c2d7ec0fba1b None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:17.732978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 590/1147] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:26:17.738407 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2025fc-b314-45e5-9a79-82f896cf0c18 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:17.738881 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2025fc-b314-45e5-9a79-82f896cf0c18 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:17.738980 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2025fc-b314-45e5-9a79-82f896cf0c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:17.758273 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-6b2025fc-b314-45e5-9a79-82f896cf0c18 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:26:17.759492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6b2025fc-b314-45e5-9a79-82f896cf0c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:26:17.823563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:17.831866 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 558/1148] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:26:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:26:18.099146 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a1a82-c15e-4571-a0d2-c2d7ec0fba1b None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:18.216568 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a1a82-c15e-4571-a0d2-c2d7ec0fba1b None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:18.245938 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:18.255187 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60ace4-f802-493e-a408-04d8e7515a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:18.271734 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60ace4-f802-493e-a408-04d8e7515a92 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:18.271734 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60ace4-f802-493e-a408-04d8e7515a92 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:18.271734 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60ace4-f802-493e-a408-04d8e7515a92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:18.271734 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e60ace4-f802-493e-a408-04d8e7515a92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:26:18.272218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-4e60ace4-f802-493e-a408-04d8e7515a92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:26:18.272218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:26:18.272218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:26:18.272218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:26:18.272218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:26:18.272218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:26:18.272218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:26:18.272218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:26:18.272218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:26:18.272218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:26:18.272218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:26:18.272218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:26:18.272218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:26:18.272218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:26:18.272218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:26:18.272218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:26:18.273541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:26:18.273541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:26:18.273541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:26:18.273541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:26:18.273541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:26:18.273541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:26:18.273541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:26:18.273541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:26:18.273541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:26:18.273541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:26:18.273541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:26:18.273541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:26:18.273541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:26:18.273541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:26:18.273541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:26:18.273541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:26:18.274489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:26:18.274489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:26:18.274489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:26:18.274489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:26:18.274489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:26:18.274489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:26:18.274489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:26:18.274489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:26:18.274489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:26:18.274489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:26:18.274489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:26:18.274489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:26:18.274489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 591/1149] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:26:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:26:18.279915 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2025fc-b314-45e5-9a79-82f896cf0c18 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:18.388471 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2025fc-b314-45e5-9a79-82f896cf0c18 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:18.437083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e07bcf-58a3-4514-98f8-e646c73f2032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:18.438207 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e07bcf-58a3-4514-98f8-e646c73f2032 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:18.438659 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e07bcf-58a3-4514-98f8-e646c73f2032 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:18.439122 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e07bcf-58a3-4514-98f8-e646c73f2032 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:18.440269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e07bcf-58a3-4514-98f8-e646c73f2032 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:26:18.440863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e07bcf-58a3-4514-98f8-e646c73f2032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:18.443317 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e07bcf-58a3-4514-98f8-e646c73f2032 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:18.460193 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 559/1150] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:26:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:18.463484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60ace4-f802-493e-a408-04d8e7515a92 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:18.575421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60ace4-f802-493e-a408-04d8e7515a92 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:18.616453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab578d-640b-40bd-848f-8c93e9bb00ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:18.617199 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab578d-640b-40bd-848f-8c93e9bb00ae None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:18.617466 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab578d-640b-40bd-848f-8c93e9bb00ae None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:18.617731 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab578d-640b-40bd-848f-8c93e9bb00ae None admin] PATH_INFO: `/v3/users/7b76dbfdaf21434bbd782603ba27f88c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:18.618997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab578d-640b-40bd-848f-8c93e9bb00ae None admin] RBAC: Authorizing `identity:get_user(user_id=7b76dbfdaf21434bbd782603ba27f88c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:18.620487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab578d-640b-40bd-848f-8c93e9bb00ae None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:18.622140 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 592/1151] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:26:18 2022] GET /identity/v3/users/7b76dbfdaf21434bbd782603ba27f88c => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:18.628132 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e07bcf-58a3-4514-98f8-e646c73f2032 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:18.689082 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:18.736550 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e07bcf-58a3-4514-98f8-e646c73f2032 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:18.783843 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87d28f-a8c2-4cd5-9103-e6abd412c72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:18.784607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87d28f-a8c2-4cd5-9103-e6abd412c72c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:18.784769 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87d28f-a8c2-4cd5-9103-e6abd412c72c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:18.785097 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87d28f-a8c2-4cd5-9103-e6abd412c72c None admin] PATH_INFO: `/v3/projects/7b3041d26908404f99d8acc5d30f4b4e` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:18.786745 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87d28f-a8c2-4cd5-9103-e6abd412c72c None admin] RBAC: Authorizing `identity:get_project(project_id=7b3041d26908404f99d8acc5d30f4b4e)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:18.788216 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87d28f-a8c2-4cd5-9103-e6abd412c72c None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:18.789951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 560/1152] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:26:18 2022] GET /identity/v3/projects/7b3041d26908404f99d8acc5d30f4b4e => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:18.796593 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab578d-640b-40bd-848f-8c93e9bb00ae None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:18.856731 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:18.902123 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab578d-640b-40bd-848f-8c93e9bb00ae None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:18.942529 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a9909-def0-4a80-b9c1-3aa17ea0ec0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:18.943105 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a9909-def0-4a80-b9c1-3aa17ea0ec0b None admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:18.943376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a9909-def0-4a80-b9c1-3aa17ea0ec0b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:18.943644 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a9909-def0-4a80-b9c1-3aa17ea0ec0b None admin] PATH_INFO: `/v3/projects/7b3041d26908404f99d8acc5d30f4b4e/users/7b76dbfdaf21434bbd782603ba27f88c/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:18.945564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a9909-def0-4a80-b9c1-3aa17ea0ec0b None admin] RBAC: Authorizing `identity:create_grant(project_id=7b3041d26908404f99d8acc5d30f4b4e, user_id=7b76dbfdaf21434bbd782603ba27f88c, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:18.946989 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a9909-def0-4a80-b9c1-3aa17ea0ec0b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:19.025983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 593/1153] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:26:18 2022] PUT /identity/v3/projects/7b3041d26908404f99d8acc5d30f4b4e/users/7b76dbfdaf21434bbd782603ba27f88c/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:26:21.009358 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dc177-1fc5-4cb2-b231-17fc50b07cca None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:21.009923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dc177-1fc5-4cb2-b231-17fc50b07cca None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:21.009923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dc177-1fc5-4cb2-b231-17fc50b07cca None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:21.011230 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 561/1154] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:26:21.017234 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5196d1-2023-4b52-b017-d222cf6204ce None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:21.017766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5196d1-2023-4b52-b017-d222cf6204ce None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:21.017766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5196d1-2023-4b52-b017-d222cf6204ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:21.038132 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-9b5196d1-2023-4b52-b017-d222cf6204ce None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:26:21.039337 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-9b5196d1-2023-4b52-b017-d222cf6204ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:26:21.120990 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 594/1155] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:26:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:26:21.400979 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271dc177-1fc5-4cb2-b231-17fc50b07cca None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:21.534706 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271dc177-1fc5-4cb2-b231-17fc50b07cca None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:21.569236 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:21.581474 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba8a1d-5ab8-4533-b6d1-ed1bd618d3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:21.583004 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba8a1d-5ab8-4533-b6d1-ed1bd618d3a5 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:21.583407 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba8a1d-5ab8-4533-b6d1-ed1bd618d3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:21.583760 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba8a1d-5ab8-4533-b6d1-ed1bd618d3a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:21.597802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba8a1d-5ab8-4533-b6d1-ed1bd618d3a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:26:21.603191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-59ba8a1d-5ab8-4533-b6d1-ed1bd618d3a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:26:21.603191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:26:21.603191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:26:21.603191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:26:21.603191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:26:21.603191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:26:21.603191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:26:21.603191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:26:21.603191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:26:21.603191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:26:21.603191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:26:21.603191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:26:21.603191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:26:21.603191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:26:21.603191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:26:21.603191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:26:21.604923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:26:21.604923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:26:21.604923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:26:21.604923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:26:21.604923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:26:21.604923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:26:21.604923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:26:21.604923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:26:21.604923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:26:21.604923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:26:21.604923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:26:21.604923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:26:21.604923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:26:21.604923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:26:21.604923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:26:21.604923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:26:21.606750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:26:21.606750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:26:21.606750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:26:21.606750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:26:21.606750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:26:21.606750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:26:21.606750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:26:21.606750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:26:21.606750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:26:21.606750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:26:21.606750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:26:21.606750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:26:21.606750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 562/1156] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:26:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:26:21.610837 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5196d1-2023-4b52-b017-d222cf6204ce None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:21.661488 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:21.721100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5196d1-2023-4b52-b017-d222cf6204ce None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:21.757656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c8d92-3c7d-4450-bb6d-663ceba50ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:21.758281 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c8d92-3c7d-4450-bb6d-663ceba50ab8 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:21.758549 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c8d92-3c7d-4450-bb6d-663ceba50ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:21.758813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c8d92-3c7d-4450-bb6d-663ceba50ab8 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:21.759467 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c8d92-3c7d-4450-bb6d-663ceba50ab8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:26:21.759755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c8d92-3c7d-4450-bb6d-663ceba50ab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:21.761146 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c8d92-3c7d-4450-bb6d-663ceba50ab8 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:21.770169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 595/1157] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:26:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:21.776348 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba8a1d-5ab8-4533-b6d1-ed1bd618d3a5 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:21.884940 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba8a1d-5ab8-4533-b6d1-ed1bd618d3a5 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:21.919631 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb845a6-6a0f-41e2-a485-391302e31d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:21.919631 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb845a6-6a0f-41e2-a485-391302e31d5f None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:21.919875 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb845a6-6a0f-41e2-a485-391302e31d5f None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:21.920155 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb845a6-6a0f-41e2-a485-391302e31d5f None admin] PATH_INFO: `/v3/users/7b76dbfdaf21434bbd782603ba27f88c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:21.921305 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb845a6-6a0f-41e2-a485-391302e31d5f None admin] RBAC: Authorizing `identity:get_user(user_id=7b76dbfdaf21434bbd782603ba27f88c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:21.922733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb845a6-6a0f-41e2-a485-391302e31d5f None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:21.924374 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 563/1158] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:26:21 2022] GET /identity/v3/users/7b76dbfdaf21434bbd782603ba27f88c => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:21.930925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c8d92-3c7d-4450-bb6d-663ceba50ab8 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:22.033692 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c8d92-3c7d-4450-bb6d-663ceba50ab8 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:22.041636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:22.069897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51cc54-87ab-4b73-9a7a-8ef634fb714c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:22.070496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51cc54-87ab-4b73-9a7a-8ef634fb714c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:22.070767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51cc54-87ab-4b73-9a7a-8ef634fb714c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:22.071030 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51cc54-87ab-4b73-9a7a-8ef634fb714c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:22.142459 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51cc54-87ab-4b73-9a7a-8ef634fb714c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:22.143914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51cc54-87ab-4b73-9a7a-8ef634fb714c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:22.251298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 596/1159] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:26:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:26:22.258316 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb845a6-6a0f-41e2-a485-391302e31d5f None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:22.322577 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:22.372773 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb845a6-6a0f-41e2-a485-391302e31d5f None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:22.407163 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc146a-8d23-4a49-b92c-eb0fec2c03b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:22.407800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc146a-8d23-4a49-b92c-eb0fec2c03b2 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:22.408073 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc146a-8d23-4a49-b92c-eb0fec2c03b2 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:22.408369 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc146a-8d23-4a49-b92c-eb0fec2c03b2 None admin] PATH_INFO: `/v3/projects/7b3041d26908404f99d8acc5d30f4b4e` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:22.409475 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc146a-8d23-4a49-b92c-eb0fec2c03b2 None admin] RBAC: Authorizing `identity:get_project(project_id=7b3041d26908404f99d8acc5d30f4b4e)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:22.410997 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc146a-8d23-4a49-b92c-eb0fec2c03b2 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:22.412724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 564/1160] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:26:22 2022] GET /identity/v3/projects/7b3041d26908404f99d8acc5d30f4b4e => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:22.418837 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51cc54-87ab-4b73-9a7a-8ef634fb714c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:22.427739 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:22.527783 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51cc54-87ab-4b73-9a7a-8ef634fb714c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:22.566936 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1bf249-1a5b-448a-a0cb-cff55a1af153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:22.567595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bf249-1a5b-448a-a0cb-cff55a1af153 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:22.567869 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bf249-1a5b-448a-a0cb-cff55a1af153 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:22.568164 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bf249-1a5b-448a-a0cb-cff55a1af153 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:22.568892 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1bf249-1a5b-448a-a0cb-cff55a1af153 None admin] RBAC: Adding query filter params (role.id=a836108447dc439e994611bc74d53d9c, scope.project.id=7b3041d26908404f99d8acc5d30f4b4e, user.id=7b76dbfdaf21434bbd782603ba27f88c) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:26:22.569190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1bf249-1a5b-448a-a0cb-cff55a1af153 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:22.570623 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1bf249-1a5b-448a-a0cb-cff55a1af153 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:22.576931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 597/1161] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:26:22 2022] GET /identity/v3/role_assignments?user.id=7b76dbfdaf21434bbd782603ba27f88c&scope.project.id=7b3041d26908404f99d8acc5d30f4b4e&role.id=a836108447dc439e994611bc74d53d9c => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:24.636475 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ad20a-cb36-49b5-946d-0c70afafcb6c None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:24.637037 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ad20a-cb36-49b5-946d-0c70afafcb6c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:24.637037 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ad20a-cb36-49b5-946d-0c70afafcb6c None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:24.638347 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 565/1162] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:26:24.643901 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f25189-0f13-405c-b6d2-566f03a246f3 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:24.645148 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f25189-0f13-405c-b6d2-566f03a246f3 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:24.645148 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f25189-0f13-405c-b6d2-566f03a246f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:24.663938 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-08f25189-0f13-405c-b6d2-566f03a246f3 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:26:24.665178 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-08f25189-0f13-405c-b6d2-566f03a246f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:26:24.735074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:24.740657 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 598/1163] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:26:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:26:25.042389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ad20a-cb36-49b5-946d-0c70afafcb6c None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:25.171606 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ad20a-cb36-49b5-946d-0c70afafcb6c None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:25.203244 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:25.226698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766a9d2-b21d-485e-973c-d0dab03c20d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:25.227402 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766a9d2-b21d-485e-973c-d0dab03c20d2 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:25.227690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766a9d2-b21d-485e-973c-d0dab03c20d2 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:25.227966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766a9d2-b21d-485e-973c-d0dab03c20d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:25.308383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766a9d2-b21d-485e-973c-d0dab03c20d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:25.311113 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766a9d2-b21d-485e-973c-d0dab03c20d2 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:25.457886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 566/1164] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:26:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:26:25.465521 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f25189-0f13-405c-b6d2-566f03a246f3 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:25.594744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f25189-0f13-405c-b6d2-566f03a246f3 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:25.650932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd2e55-81b2-48b3-86d8-90f778ead097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:25.651636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd2e55-81b2-48b3-86d8-90f778ead097 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:25.651826 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd2e55-81b2-48b3-86d8-90f778ead097 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:25.652090 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd2e55-81b2-48b3-86d8-90f778ead097 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:25.659817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd2e55-81b2-48b3-86d8-90f778ead097 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:25.661848 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd2e55-81b2-48b3-86d8-90f778ead097 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:25.677949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-a5cd2e55-81b2-48b3-86d8-90f778ead097 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 22:26:25.677949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:26:25.677949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 29 22:26:25.677949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 29 22:26:25.677949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:26:25.677949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:26:25.677949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:26:25.677949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:26:25.677949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 29 22:26:25.677949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:26:25.677949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:26:25.677949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:26:25.677949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:26:25.677949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:26:25.677949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:26:25.677949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:26:25.677949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:26:25.679511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:26:25.679511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:26:25.679511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:26:25.679511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:26:25.679511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:26:25.679511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:26:25.679511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:26:25.679511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:26:25.679511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:26:25.679511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:26:25.679511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:26:25.679511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:26:25.679511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:26:25.679511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:26:25.679511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:26:25.679511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:26:25.681273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:26:25.681273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:26:25.681273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:26:25.681273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:26:25.681273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:26:25.681273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:26:25.681273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:26:25.681273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:26:25.681273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:26:25.681273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:26:25.681273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 29 22:26:25.681273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:26:25.681273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 22:26:25.681273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:26:25.681273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 599/1165] 10.222.0.59 () {56 vars in 1117 bytes} [Mon Aug 29 22:26:25 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:26:25.686736 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766a9d2-b21d-485e-973c-d0dab03c20d2 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:25.713530 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:25.820094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766a9d2-b21d-485e-973c-d0dab03c20d2 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:25.870331 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a8ce21-4922-4c57-ab41-47351a4da273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:25.870942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8ce21-4922-4c57-ab41-47351a4da273 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:25.871218 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8ce21-4922-4c57-ab41-47351a4da273 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:25.871488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8ce21-4922-4c57-ab41-47351a4da273 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:25.872179 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a8ce21-4922-4c57-ab41-47351a4da273 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:26:25.872470 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a8ce21-4922-4c57-ab41-47351a4da273 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:25.873834 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a8ce21-4922-4c57-ab41-47351a4da273 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:25.883139 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 567/1166] 10.222.0.59 () {56 vars in 1115 bytes} [Mon Aug 29 22:26:25 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:27.880105 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a6d3-0d2d-4937-9bc6-f4df92e9a89f None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:27.881418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a6d3-0d2d-4937-9bc6-f4df92e9a89f None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:27.881418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a6d3-0d2d-4937-9bc6-f4df92e9a89f None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:27.882602 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 600/1167] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:26:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:26:27.889042 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41f6cc-f793-46ca-824d-dabd1b877a7b None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:27.889569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41f6cc-f793-46ca-824d-dabd1b877a7b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:27.889660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41f6cc-f793-46ca-824d-dabd1b877a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:27.908882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-9f41f6cc-f793-46ca-824d-dabd1b877a7b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:26:27.910835 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-9f41f6cc-f793-46ca-824d-dabd1b877a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:26:27.999150 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 568/1168] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:26:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:26:28.283978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162a6d3-0d2d-4937-9bc6-f4df92e9a89f None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:28.355025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:28.404851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162a6d3-0d2d-4937-9bc6-f4df92e9a89f None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:28.442381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b862a0f-57ab-4264-889b-b1e16a84316c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:28.442990 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b862a0f-57ab-4264-889b-b1e16a84316c None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:28.443268 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b862a0f-57ab-4264-889b-b1e16a84316c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:28.443536 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b862a0f-57ab-4264-889b-b1e16a84316c None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:28.444224 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b862a0f-57ab-4264-889b-b1e16a84316c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:28.445601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b862a0f-57ab-4264-889b-b1e16a84316c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:28.518089 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 601/1169] 10.222.0.59 () {60 vars in 1138 bytes} [Mon Aug 29 22:26:28 2022] POST /identity/v3/domains => generated 267 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:26:30.600128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb181a-a3fb-4ad6-97a1-ab41537ea402 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:30.600727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb181a-a3fb-4ad6-97a1-ab41537ea402 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:30.600727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb181a-a3fb-4ad6-97a1-ab41537ea402 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:30.602059 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 569/1170] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:26:30.607489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ab128-2804-44b2-b0f4-17c57c07780b None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:30.607925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ab128-2804-44b2-b0f4-17c57c07780b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:30.608131 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ab128-2804-44b2-b0f4-17c57c07780b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:30.627132 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-c86ab128-2804-44b2-b0f4-17c57c07780b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:26:30.628977 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c86ab128-2804-44b2-b0f4-17c57c07780b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:26:30.699060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 602/1171] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:26:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:26:30.978209 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb181a-a3fb-4ad6-97a1-ab41537ea402 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:30.995244 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:31.089695 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb181a-a3fb-4ad6-97a1-ab41537ea402 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:31.128373 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024e46e-d62b-48a0-8d73-69575eddc9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:31.129064 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024e46e-d62b-48a0-8d73-69575eddc9e6 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:31.129600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024e46e-d62b-48a0-8d73-69575eddc9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:31.129600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024e46e-d62b-48a0-8d73-69575eddc9e6 None admin] PATH_INFO: `/v3/domains/76468e68d5b845caac1ddc4becd0e0d4` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:31.140756 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0024e46e-d62b-48a0-8d73-69575eddc9e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=76468e68d5b845caac1ddc4becd0e0d4)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:31.142247 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0024e46e-d62b-48a0-8d73-69575eddc9e6 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:31.143960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 570/1172] 10.222.0.59 () {56 vars in 1183 bytes} [Mon Aug 29 22:26:30 2022] GET /identity/v3/domains/76468e68d5b845caac1ddc4becd0e0d4 => generated 267 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:31.149904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ab128-2804-44b2-b0f4-17c57c07780b None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:31.210275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:31.262318 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ab128-2804-44b2-b0f4-17c57c07780b None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:31.298849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d56e6-87c4-4a8f-a6f8-d990c7dd5658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:31.299463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d56e6-87c4-4a8f-a6f8-d990c7dd5658 None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:31.300303 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d56e6-87c4-4a8f-a6f8-d990c7dd5658 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:31.300578 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d56e6-87c4-4a8f-a6f8-d990c7dd5658 None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:31.301229 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d56e6-87c4-4a8f-a6f8-d990c7dd5658 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:31.302679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d56e6-87c4-4a8f-a6f8-d990c7dd5658 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:31.387676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 603/1173] 10.222.0.59 () {60 vars in 1141 bytes} [Mon Aug 29 22:26:31 2022] POST /identity/v3/projects => generated 361 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:26:33.389720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ace19f-c9b4-4de6-8ba9-e0ce8127b0ff None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:33.390257 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ace19f-c9b4-4de6-8ba9-e0ce8127b0ff None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:33.390257 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ace19f-c9b4-4de6-8ba9-e0ce8127b0ff None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:33.391591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 571/1174] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:26:33.397476 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a8484-104f-4987-8942-ba9c43aec759 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:33.397887 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a8484-104f-4987-8942-ba9c43aec759 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:33.398024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a8484-104f-4987-8942-ba9c43aec759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:33.416153 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-cd5a8484-104f-4987-8942-ba9c43aec759 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:26:33.417367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-cd5a8484-104f-4987-8942-ba9c43aec759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:26:33.485008 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 604/1175] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:26:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:26:33.757521 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ace19f-c9b4-4de6-8ba9-e0ce8127b0ff None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:33.837729 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:33.864419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ace19f-c9b4-4de6-8ba9-e0ce8127b0ff None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:33.900472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bcdfce-1eb9-4d45-8482-b389c5188206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:33.901108 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcdfce-1eb9-4d45-8482-b389c5188206 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:33.901383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcdfce-1eb9-4d45-8482-b389c5188206 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:33.901652 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcdfce-1eb9-4d45-8482-b389c5188206 None admin] PATH_INFO: `/v3/domains/76468e68d5b845caac1ddc4becd0e0d4` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:33.902776 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bcdfce-1eb9-4d45-8482-b389c5188206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=76468e68d5b845caac1ddc4becd0e0d4)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:33.904236 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bcdfce-1eb9-4d45-8482-b389c5188206 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:33.905818 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 572/1176] 10.222.0.59 () {56 vars in 1183 bytes} [Mon Aug 29 22:26:33 2022] GET /identity/v3/domains/76468e68d5b845caac1ddc4becd0e0d4 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:33.912032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a8484-104f-4987-8942-ba9c43aec759 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:33.945910 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:34.019507 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a8484-104f-4987-8942-ba9c43aec759 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:34.054705 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3c8c7-62ca-4842-809d-3c0444784085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:34.055309 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3c8c7-62ca-4842-809d-3c0444784085 None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:34.055587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3c8c7-62ca-4842-809d-3c0444784085 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:34.055855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3c8c7-62ca-4842-809d-3c0444784085 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:34.056541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3c8c7-62ca-4842-809d-3c0444784085 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:34.057971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3c8c7-62ca-4842-809d-3c0444784085 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:34.168053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 605/1177] 10.222.0.59 () {60 vars in 1132 bytes} [Mon Aug 29 22:26:33 2022] POST /identity/v3/users => generated 311 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:26:36.216511 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e9f8f-d1b7-4db9-94e0-80f612be17cb None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:36.217022 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e9f8f-d1b7-4db9-94e0-80f612be17cb None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:36.217022 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e9f8f-d1b7-4db9-94e0-80f612be17cb None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:36.218302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 573/1178] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:26:36.224393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568aa0f-9af6-4a64-99c4-41284c2ff3ea None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:36.224912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568aa0f-9af6-4a64-99c4-41284c2ff3ea None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:36.224912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568aa0f-9af6-4a64-99c4-41284c2ff3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:36.244584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-2568aa0f-9af6-4a64-99c4-41284c2ff3ea None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:26:36.246128 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-2568aa0f-9af6-4a64-99c4-41284c2ff3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:26:36.324601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 606/1179] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:26:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:26:36.591492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e9f8f-d1b7-4db9-94e0-80f612be17cb None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:36.698084 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e9f8f-d1b7-4db9-94e0-80f612be17cb None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:36.733031 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37a048-4156-42a9-b444-20c2359309c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:36.733663 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37a048-4156-42a9-b444-20c2359309c0 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:36.733940 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37a048-4156-42a9-b444-20c2359309c0 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:36.734207 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37a048-4156-42a9-b444-20c2359309c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:36.745329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37a048-4156-42a9-b444-20c2359309c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:26:36.750164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-be37a048-4156-42a9-b444-20c2359309c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:26:36.750164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:26:36.750164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:26:36.750164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:26:36.750164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:26:36.750164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:26:36.750164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:26:36.750164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:26:36.750164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:26:36.750164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:26:36.750164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:26:36.750164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:26:36.750164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:26:36.750164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:26:36.750164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:26:36.750164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:26:36.751569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:26:36.751569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:26:36.751569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:26:36.751569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:26:36.751569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:26:36.751569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:26:36.751569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:26:36.751569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:26:36.751569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:26:36.751569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:26:36.751569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:26:36.751569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:26:36.751569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:26:36.751569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:26:36.751569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:26:36.751569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:26:36.752770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:26:36.752770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:26:36.752770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:26:36.752770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:26:36.752770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:26:36.752770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:26:36.752770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:26:36.752770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:26:36.752770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:26:36.752770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:26:36.752770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:26:36.752770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:26:36.752770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 574/1180] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:26:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:26:36.757584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568aa0f-9af6-4a64-99c4-41284c2ff3ea None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:36.776272 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:36.856732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568aa0f-9af6-4a64-99c4-41284c2ff3ea None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:36.896387 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01920e-810f-4d1e-a706-640502871882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:36.896986 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01920e-810f-4d1e-a706-640502871882 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:36.897256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01920e-810f-4d1e-a706-640502871882 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:36.897521 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01920e-810f-4d1e-a706-640502871882 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:36.898167 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01920e-810f-4d1e-a706-640502871882 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:26:36.898459 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01920e-810f-4d1e-a706-640502871882 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:36.899813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01920e-810f-4d1e-a706-640502871882 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:36.908156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 607/1181] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:26:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:36.914870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37a048-4156-42a9-b444-20c2359309c0 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:36.917877 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:37.016890 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37a048-4156-42a9-b444-20c2359309c0 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:37.051324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31169f64-d581-4f75-af72-0d1d717f5e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:37.051942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31169f64-d581-4f75-af72-0d1d717f5e44 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:37.052245 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31169f64-d581-4f75-af72-0d1d717f5e44 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:37.052514 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31169f64-d581-4f75-af72-0d1d717f5e44 None admin] PATH_INFO: `/v3/users/2dbc31091571448a8844171ec543bcfa` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:37.065324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31169f64-d581-4f75-af72-0d1d717f5e44 None admin] RBAC: Authorizing `identity:get_user(user_id=2dbc31091571448a8844171ec543bcfa)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:37.067029 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31169f64-d581-4f75-af72-0d1d717f5e44 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:37.068732 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 575/1182] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:26:36 2022] GET /identity/v3/users/2dbc31091571448a8844171ec543bcfa => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:37.074808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01920e-810f-4d1e-a706-640502871882 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:37.159879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:37.180264 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01920e-810f-4d1e-a706-640502871882 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:37.215040 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7830d-d2c5-4bd7-971b-9520a90b3144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:37.215638 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7830d-d2c5-4bd7-971b-9520a90b3144 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:37.215916 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7830d-d2c5-4bd7-971b-9520a90b3144 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:37.216211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7830d-d2c5-4bd7-971b-9520a90b3144 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:37.284308 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7830d-d2c5-4bd7-971b-9520a90b3144 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:37.285762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7830d-d2c5-4bd7-971b-9520a90b3144 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:37.364534 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:37.383890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 608/1183] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:26:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:26:37.390190 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31169f64-d581-4f75-af72-0d1d717f5e44 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:37.449359 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:37.488395 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31169f64-d581-4f75-af72-0d1d717f5e44 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:37.522854 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4775f80-35bc-4ae8-b176-840a3c6be5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:37.523474 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4775f80-35bc-4ae8-b176-840a3c6be5b4 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:37.523748 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4775f80-35bc-4ae8-b176-840a3c6be5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:37.524014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4775f80-35bc-4ae8-b176-840a3c6be5b4 None admin] PATH_INFO: `/v3/projects/c5fbf5c3a6d54342a8ee70b19ba16fb4` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:37.525153 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4775f80-35bc-4ae8-b176-840a3c6be5b4 None admin] RBAC: Authorizing `identity:get_project(project_id=c5fbf5c3a6d54342a8ee70b19ba16fb4)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:37.526653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4775f80-35bc-4ae8-b176-840a3c6be5b4 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:37.528291 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 576/1184] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:26:37 2022] GET /identity/v3/projects/c5fbf5c3a6d54342a8ee70b19ba16fb4 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:37.534492 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7830d-d2c5-4bd7-971b-9520a90b3144 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:37.640893 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7830d-d2c5-4bd7-971b-9520a90b3144 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:37.681311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b336e-cd1c-414c-b384-b9f6411e8a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:37.681914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b336e-cd1c-414c-b384-b9f6411e8a97 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:37.682192 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b336e-cd1c-414c-b384-b9f6411e8a97 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:37.682460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b336e-cd1c-414c-b384-b9f6411e8a97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:37.683198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b336e-cd1c-414c-b384-b9f6411e8a97 None admin] RBAC: Adding query filter params (role.id=a836108447dc439e994611bc74d53d9c, scope.project.id=c5fbf5c3a6d54342a8ee70b19ba16fb4, user.id=2dbc31091571448a8844171ec543bcfa) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:26:37.683502 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b336e-cd1c-414c-b384-b9f6411e8a97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:37.684973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b336e-cd1c-414c-b384-b9f6411e8a97 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:37.691844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 609/1185] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:26:37 2022] GET /identity/v3/role_assignments?user.id=2dbc31091571448a8844171ec543bcfa&scope.project.id=c5fbf5c3a6d54342a8ee70b19ba16fb4&role.id=a836108447dc439e994611bc74d53d9c => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:39.612382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c208e7-50c0-40e6-930c-5cc119ba5386 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:39.613150 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c208e7-50c0-40e6-930c-5cc119ba5386 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:39.613150 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c208e7-50c0-40e6-930c-5cc119ba5386 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:39.614811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 577/1186] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:26:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:26:39.621227 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a541d-bc2a-403f-a46a-b0d0005769db None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:39.621673 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a541d-bc2a-403f-a46a-b0d0005769db None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:39.621791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a541d-bc2a-403f-a46a-b0d0005769db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:39.640399 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-3e1a541d-bc2a-403f-a46a-b0d0005769db None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:26:39.641589 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3e1a541d-bc2a-403f-a46a-b0d0005769db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:26:39.683002 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:39.710154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 610/1187] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:26:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:26:39.982445 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c208e7-50c0-40e6-930c-5cc119ba5386 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:40.085317 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c208e7-50c0-40e6-930c-5cc119ba5386 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:40.111683 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:40.120234 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb648f7d-1f4b-4b33-b827-a1759f515302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:40.126201 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb648f7d-1f4b-4b33-b827-a1759f515302 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:40.126201 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb648f7d-1f4b-4b33-b827-a1759f515302 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:40.126201 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb648f7d-1f4b-4b33-b827-a1759f515302 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:40.131550 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb648f7d-1f4b-4b33-b827-a1759f515302 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:26:40.136740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-fb648f7d-1f4b-4b33-b827-a1759f515302 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:26:40.136740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:26:40.136740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:26:40.136740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:26:40.136740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:26:40.136740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:26:40.136740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:26:40.136740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:26:40.136740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:26:40.136740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:26:40.136740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:26:40.136740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:26:40.136740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:26:40.136740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:26:40.136740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:26:40.136740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:26:40.137828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:26:40.137828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:26:40.137828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:26:40.137828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:26:40.137828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:26:40.137828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:26:40.137828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:26:40.137828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:26:40.137828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:26:40.137828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:26:40.137828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:26:40.137828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:26:40.137828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:26:40.137828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:26:40.137828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:26:40.137828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:26:40.141243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:26:40.141243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:26:40.141243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:26:40.141243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:26:40.141243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:26:40.141243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:26:40.141243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:26:40.141243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:26:40.141243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:26:40.141243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:26:40.141243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:26:40.141243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:26:40.141243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 578/1188] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:26:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:26:40.143979 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a541d-bc2a-403f-a46a-b0d0005769db None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:40.244557 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a541d-bc2a-403f-a46a-b0d0005769db None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:40.279109 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da413f72-cd6b-4082-9401-b223ec1ef9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:40.279711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da413f72-cd6b-4082-9401-b223ec1ef9c0 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:40.279988 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da413f72-cd6b-4082-9401-b223ec1ef9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:40.280294 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da413f72-cd6b-4082-9401-b223ec1ef9c0 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:40.280956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da413f72-cd6b-4082-9401-b223ec1ef9c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:26:40.281256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da413f72-cd6b-4082-9401-b223ec1ef9c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:40.282626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da413f72-cd6b-4082-9401-b223ec1ef9c0 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:40.290826 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 611/1189] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:26:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:40.296699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb648f7d-1f4b-4b33-b827-a1759f515302 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:40.402482 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb648f7d-1f4b-4b33-b827-a1759f515302 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:40.436389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1b517-7867-4819-b650-c3aca1229905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:40.437010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1b517-7867-4819-b650-c3aca1229905 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:40.437285 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1b517-7867-4819-b650-c3aca1229905 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:40.437553 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1b517-7867-4819-b650-c3aca1229905 None admin] PATH_INFO: `/v3/users/2dbc31091571448a8844171ec543bcfa` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:40.438665 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1b517-7867-4819-b650-c3aca1229905 None admin] RBAC: Authorizing `identity:get_user(user_id=2dbc31091571448a8844171ec543bcfa)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:40.440757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1b517-7867-4819-b650-c3aca1229905 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:40.442355 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 579/1190] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:26:40 2022] GET /identity/v3/users/2dbc31091571448a8844171ec543bcfa => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:40.448440 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da413f72-cd6b-4082-9401-b223ec1ef9c0 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:40.484084 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:40.550421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da413f72-cd6b-4082-9401-b223ec1ef9c0 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:40.586382 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f0513-f2da-471b-a500-ad5757622dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:40.587007 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f0513-f2da-471b-a500-ad5757622dcd None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:40.587286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f0513-f2da-471b-a500-ad5757622dcd None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:40.587556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f0513-f2da-471b-a500-ad5757622dcd None admin] PATH_INFO: `/v3/projects/c5fbf5c3a6d54342a8ee70b19ba16fb4` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:40.588704 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f0513-f2da-471b-a500-ad5757622dcd None admin] RBAC: Authorizing `identity:get_project(project_id=c5fbf5c3a6d54342a8ee70b19ba16fb4)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:40.590135 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f0513-f2da-471b-a500-ad5757622dcd None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:40.591769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 612/1191] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:26:40 2022] GET /identity/v3/projects/c5fbf5c3a6d54342a8ee70b19ba16fb4 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:40.598947 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1b517-7867-4819-b650-c3aca1229905 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:40.654942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:40.699898 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1b517-7867-4819-b650-c3aca1229905 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:40.735745 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11907070-c40b-4860-a1dc-abd7e7c3fa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:40.736442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11907070-c40b-4860-a1dc-abd7e7c3fa93 None admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:40.736804 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11907070-c40b-4860-a1dc-abd7e7c3fa93 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:40.737262 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11907070-c40b-4860-a1dc-abd7e7c3fa93 None admin] PATH_INFO: `/v3/projects/c5fbf5c3a6d54342a8ee70b19ba16fb4/users/2dbc31091571448a8844171ec543bcfa/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:40.739646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11907070-c40b-4860-a1dc-abd7e7c3fa93 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5fbf5c3a6d54342a8ee70b19ba16fb4, user_id=2dbc31091571448a8844171ec543bcfa, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:40.741110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11907070-c40b-4860-a1dc-abd7e7c3fa93 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:40.807109 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 580/1192] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:26:40 2022] PUT /identity/v3/projects/c5fbf5c3a6d54342a8ee70b19ba16fb4/users/2dbc31091571448a8844171ec543bcfa/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:26:42.737953 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea910a6-a0d9-4d19-99b8-f62b61d654f0 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:42.738750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea910a6-a0d9-4d19-99b8-f62b61d654f0 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:42.738750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea910a6-a0d9-4d19-99b8-f62b61d654f0 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:42.740119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 613/1193] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:26:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:26:42.745585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f2a6b-0d45-4d2e-ac15-cc044d2e7cad None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:42.746029 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f2a6b-0d45-4d2e-ac15-cc044d2e7cad None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:42.746121 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f2a6b-0d45-4d2e-ac15-cc044d2e7cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:42.764800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-c66f2a6b-0d45-4d2e-ac15-cc044d2e7cad None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:26:42.765997 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-c66f2a6b-0d45-4d2e-ac15-cc044d2e7cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:26:42.833244 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 581/1194] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:26:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:26:43.109597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea910a6-a0d9-4d19-99b8-f62b61d654f0 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:43.219280 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea910a6-a0d9-4d19-99b8-f62b61d654f0 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:43.222189 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:43.255445 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bee810-4138-469d-a564-53ba7c921835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:43.256160 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bee810-4138-469d-a564-53ba7c921835 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:43.256455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bee810-4138-469d-a564-53ba7c921835 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:43.256785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bee810-4138-469d-a564-53ba7c921835 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:43.267575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bee810-4138-469d-a564-53ba7c921835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:26:43.273169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-69bee810-4138-469d-a564-53ba7c921835 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:26:43.273169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:26:43.273169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:26:43.273169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:26:43.273169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:26:43.273169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:26:43.273169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:26:43.273169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:26:43.273169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:26:43.273169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:26:43.273169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:26:43.273169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:26:43.273169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:26:43.273169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:26:43.273169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:26:43.273169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:26:43.274344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:26:43.274344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:26:43.274344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:26:43.274344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:26:43.274344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:26:43.274344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:26:43.274344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:26:43.274344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:26:43.274344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:26:43.274344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:26:43.274344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:26:43.274344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:26:43.274344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:26:43.274344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:26:43.274344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:26:43.274344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:26:43.275203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:26:43.275203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:26:43.275203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:26:43.275203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:26:43.275203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:26:43.275203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:26:43.275203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:26:43.275203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:26:43.275203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:26:43.275203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:26:43.275203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:26:43.275203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:26:43.275203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 614/1195] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:26:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:26:43.280401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f2a6b-0d45-4d2e-ac15-cc044d2e7cad None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:43.328202 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:43.382965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f2a6b-0d45-4d2e-ac15-cc044d2e7cad None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:43.418130 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e296a-fb55-425d-ad8f-11a74d2a4ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:43.418735 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e296a-fb55-425d-ad8f-11a74d2a4ea3 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:43.419014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e296a-fb55-425d-ad8f-11a74d2a4ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:43.419286 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e296a-fb55-425d-ad8f-11a74d2a4ea3 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:43.419944 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e296a-fb55-425d-ad8f-11a74d2a4ea3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:26:43.420273 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e296a-fb55-425d-ad8f-11a74d2a4ea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:43.421643 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e296a-fb55-425d-ad8f-11a74d2a4ea3 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:43.430435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 582/1196] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:26:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:43.436292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bee810-4138-469d-a564-53ba7c921835 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:43.538001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bee810-4138-469d-a564-53ba7c921835 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:43.572777 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a4479-6b3a-4117-aa2e-7ff37959a22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:43.573393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a4479-6b3a-4117-aa2e-7ff37959a22d None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:43.573664 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a4479-6b3a-4117-aa2e-7ff37959a22d None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:43.573932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a4479-6b3a-4117-aa2e-7ff37959a22d None admin] PATH_INFO: `/v3/users/2dbc31091571448a8844171ec543bcfa` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:43.575029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a4479-6b3a-4117-aa2e-7ff37959a22d None admin] RBAC: Authorizing `identity:get_user(user_id=2dbc31091571448a8844171ec543bcfa)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:43.576467 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a4479-6b3a-4117-aa2e-7ff37959a22d None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:43.578026 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 615/1197] 10.222.0.59 () {56 vars in 1177 bytes} [Mon Aug 29 22:26:43 2022] GET /identity/v3/users/2dbc31091571448a8844171ec543bcfa => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:43.585426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e296a-fb55-425d-ad8f-11a74d2a4ea3 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:43.685811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e296a-fb55-425d-ad8f-11a74d2a4ea3 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:43.694840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:43.723060 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18b809-d752-4ef6-9572-aa46fc9f630e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:43.723666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18b809-d752-4ef6-9572-aa46fc9f630e None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:43.723939 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18b809-d752-4ef6-9572-aa46fc9f630e None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:43.724235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18b809-d752-4ef6-9572-aa46fc9f630e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:43.794769 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd18b809-d752-4ef6-9572-aa46fc9f630e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:43.796241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd18b809-d752-4ef6-9572-aa46fc9f630e None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:43.900151 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 583/1198] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:26:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:26:43.906386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a4479-6b3a-4117-aa2e-7ff37959a22d None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:43.938042 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:44.008280 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a4479-6b3a-4117-aa2e-7ff37959a22d None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:44.042213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3162a-606b-4e68-a168-11023f0b9cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:44.042829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3162a-606b-4e68-a168-11023f0b9cce None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:44.043104 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3162a-606b-4e68-a168-11023f0b9cce None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:44.043377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3162a-606b-4e68-a168-11023f0b9cce None admin] PATH_INFO: `/v3/projects/c5fbf5c3a6d54342a8ee70b19ba16fb4` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:44.044662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3162a-606b-4e68-a168-11023f0b9cce None admin] RBAC: Authorizing `identity:get_project(project_id=c5fbf5c3a6d54342a8ee70b19ba16fb4)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:44.046083 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3162a-606b-4e68-a168-11023f0b9cce None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:44.047710 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 616/1199] 10.222.0.59 () {56 vars in 1186 bytes} [Mon Aug 29 22:26:43 2022] GET /identity/v3/projects/c5fbf5c3a6d54342a8ee70b19ba16fb4 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:44.053903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18b809-d752-4ef6-9572-aa46fc9f630e None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:26:44.064005 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:44.155774 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18b809-d752-4ef6-9572-aa46fc9f630e None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:26:44.189651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee67bd-06b1-4e86-9a0e-d577337379aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:26:44.190191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee67bd-06b1-4e86-9a0e-d577337379aa None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:26:44.190465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee67bd-06b1-4e86-9a0e-d577337379aa None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:26:44.190730 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee67bd-06b1-4e86-9a0e-d577337379aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:26:44.191936 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee67bd-06b1-4e86-9a0e-d577337379aa None admin] RBAC: Adding query filter params (role.id=a836108447dc439e994611bc74d53d9c, scope.project.id=c5fbf5c3a6d54342a8ee70b19ba16fb4, user.id=2dbc31091571448a8844171ec543bcfa) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:26:44.192262 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee67bd-06b1-4e86-9a0e-d577337379aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:26:44.193679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee67bd-06b1-4e86-9a0e-d577337379aa None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:26:44.199416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 584/1200] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:26:44 2022] GET /identity/v3/role_assignments?user.id=2dbc31091571448a8844171ec543bcfa&scope.project.id=c5fbf5c3a6d54342a8ee70b19ba16fb4&role.id=a836108447dc439e994611bc74d53d9c => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:26:54.038004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=50 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:26:54.196873 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=64 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:58.725530 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1b685-6fea-412d-ac03-b3ec7584687c None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:58.726123 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1b685-6fea-412d-ac03-b3ec7584687c None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:58.726123 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1b685-6fea-412d-ac03-b3ec7584687c None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:58.727390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 617/1201] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:27:58.732746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d182dc-f79a-4a6c-af9d-f5fc8e8199e2 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:58.739714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d182dc-f79a-4a6c-af9d-f5fc8e8199e2 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:58.739714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d182dc-f79a-4a6c-af9d-f5fc8e8199e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:58.752492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-61d182dc-f79a-4a6c-af9d-f5fc8e8199e2 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:27:58.754430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-61d182dc-f79a-4a6c-af9d-f5fc8e8199e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:27:58.822347 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 585/1202] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:27:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:27:59.101508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1b685-6fea-412d-ac03-b3ec7584687c None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:59.209986 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1b685-6fea-412d-ac03-b3ec7584687c None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:59.244533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64faa8c5-0068-4f46-a500-a608d7efb29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:59.245161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64faa8c5-0068-4f46-a500-a608d7efb29b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:59.245439 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64faa8c5-0068-4f46-a500-a608d7efb29b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:59.245710 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64faa8c5-0068-4f46-a500-a608d7efb29b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:59.256529 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64faa8c5-0068-4f46-a500-a608d7efb29b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:59.257976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64faa8c5-0068-4f46-a500-a608d7efb29b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:59.272898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-64faa8c5-0068-4f46-a500-a608d7efb29b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:27:59.272898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:27:59.272898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:27:59.272898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:27:59.272898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:27:59.272898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:27:59.272898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:27:59.272898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:27:59.272898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:27:59.272898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:27:59.272898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:27:59.272898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:27:59.272898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:27:59.272898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:27:59.272898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:27:59.272898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:27:59.274144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:27:59.274144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:27:59.274144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:27:59.274144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:27:59.274144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:27:59.274144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:27:59.274144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:27:59.274144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:27:59.274144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:27:59.274144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:27:59.274144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:27:59.274144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:27:59.274144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:27:59.274144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:27:59.274144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:27:59.274144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:27:59.274144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:27:59.275320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:27:59.275320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:27:59.275320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:27:59.275320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 618/1203] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:27:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:27:59.280230 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d182dc-f79a-4a6c-af9d-f5fc8e8199e2 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:59.383657 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d182dc-f79a-4a6c-af9d-f5fc8e8199e2 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:59.415131 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:59.422213 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd891d1-c936-4637-a9b3-822c8d7b2c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:59.422895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd891d1-c936-4637-a9b3-822c8d7b2c42 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:59.423207 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd891d1-c936-4637-a9b3-822c8d7b2c42 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:59.423523 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd891d1-c936-4637-a9b3-822c8d7b2c42 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:59.424261 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd891d1-c936-4637-a9b3-822c8d7b2c42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:27:59.424623 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd891d1-c936-4637-a9b3-822c8d7b2c42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:59.426693 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd891d1-c936-4637-a9b3-822c8d7b2c42 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:59.438551 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 586/1204] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:27:59.445535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64faa8c5-0068-4f46-a500-a608d7efb29b None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:27:59.499778 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:27:59.549232 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64faa8c5-0068-4f46-a500-a608d7efb29b None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:27:59.583460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750cf46-a126-49b0-80e2-8adaf6585b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:27:59.584082 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750cf46-a126-49b0-80e2-8adaf6585b27 None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:27:59.584386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750cf46-a126-49b0-80e2-8adaf6585b27 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:27:59.584654 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750cf46-a126-49b0-80e2-8adaf6585b27 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:27:59.585286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750cf46-a126-49b0-80e2-8adaf6585b27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:27:59.586702 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750cf46-a126-49b0-80e2-8adaf6585b27 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:27:59.683998 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 619/1205] 10.222.0.59 () {60 vars in 1132 bytes} [Mon Aug 29 22:27:59 2022] POST /identity/v3/users => generated 251 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:28:01.645745 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a61d5-d286-411d-9aff-5ac18128dc1b None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:01.646297 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a61d5-d286-411d-9aff-5ac18128dc1b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:01.646297 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a61d5-d286-411d-9aff-5ac18128dc1b None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:01.647620 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 587/1206] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:01.652834 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877a57d-485c-4f14-937f-9356715d3cbf None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:01.653281 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877a57d-485c-4f14-937f-9356715d3cbf None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:01.653418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877a57d-485c-4f14-937f-9356715d3cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:01.672382 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-d877a57d-485c-4f14-937f-9356715d3cbf None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:01.673587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-d877a57d-485c-4f14-937f-9356715d3cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:01.745264 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 620/1207] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:28:02.018535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a61d5-d286-411d-9aff-5ac18128dc1b None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:02.121211 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a61d5-d286-411d-9aff-5ac18128dc1b None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:02.155131 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27d013-343b-4ba6-9ad7-413573cbd971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:02.155758 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27d013-343b-4ba6-9ad7-413573cbd971 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:02.156035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27d013-343b-4ba6-9ad7-413573cbd971 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:02.156357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27d013-343b-4ba6-9ad7-413573cbd971 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:02.167487 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27d013-343b-4ba6-9ad7-413573cbd971 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:02.172516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-6f27d013-343b-4ba6-9ad7-413573cbd971 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:28:02.172516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:02.172516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:02.172516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:02.172516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:02.172516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:02.172516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:02.172516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:02.172516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:02.172516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:02.172516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:02.172516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:02.172516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:02.172516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:02.172516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:02.172516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:02.173526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:02.173526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:02.173526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:02.173526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:02.173526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:02.173526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:02.173526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:02.173526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:02.173526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:02.173526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:02.173526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:02.173526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:02.173526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:02.173526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:02.173526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:02.173526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:02.174667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:02.174667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:02.174667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:02.174667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:02.174667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:02.174667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:02.174667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:02.174667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:02.174667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:02.174667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:02.174667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:28:02.174667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:02.174667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 588/1208] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:28:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:02.179859 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877a57d-485c-4f14-937f-9356715d3cbf None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:02.223998 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:02.286099 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877a57d-485c-4f14-937f-9356715d3cbf None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:02.319347 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de582d-914f-4a2b-a343-87060b810e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:02.319943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de582d-914f-4a2b-a343-87060b810e12 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:02.320242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de582d-914f-4a2b-a343-87060b810e12 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:02.320508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de582d-914f-4a2b-a343-87060b810e12 None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:02.321157 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de582d-914f-4a2b-a343-87060b810e12 None admin] RBAC: Adding query filter params (name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:02.321449 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de582d-914f-4a2b-a343-87060b810e12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:02.322801 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de582d-914f-4a2b-a343-87060b810e12 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:02.330910 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 621/1209] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:28:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:02.337019 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27d013-343b-4ba6-9ad7-413573cbd971 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:02.389840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:02.439834 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27d013-343b-4ba6-9ad7-413573cbd971 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:02.474044 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2241b19-ca5a-47d8-8894-7b6067e0af1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:02.474666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2241b19-ca5a-47d8-8894-7b6067e0af1c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:02.474943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2241b19-ca5a-47d8-8894-7b6067e0af1c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:02.475226 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2241b19-ca5a-47d8-8894-7b6067e0af1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:02.486777 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2241b19-ca5a-47d8-8894-7b6067e0af1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:02.488938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2241b19-ca5a-47d8-8894-7b6067e0af1c None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:02.503280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-d2241b19-ca5a-47d8-8894-7b6067e0af1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:02.503280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:02.503280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:02.503280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:02.503280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:02.503280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:02.503280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:02.503280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:02.503280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:02.503280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:02.503280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:02.503280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:02.503280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:28:02.503280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:28:02.503280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:28:02.503280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:28:02.506308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:02.506308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:02.506308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:02.506308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:02.506308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:02.506308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:02.506308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:02.506308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:02.506308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:02.506308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:02.506308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:02.506308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:02.506308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:02.506308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:02.506308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:02.506308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:02.506308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:28:02.507202 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:28:02.507202 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:02.507202 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:02.507202 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 589/1210] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:02.510818 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de582d-914f-4a2b-a343-87060b810e12 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:02.610613 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:02.614191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de582d-914f-4a2b-a343-87060b810e12 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:02.648669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f44e9-e709-4b38-a10b-98bb7a885446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:02.649417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f44e9-e709-4b38-a10b-98bb7a885446 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:02.649751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f44e9-e709-4b38-a10b-98bb7a885446 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:02.650322 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f44e9-e709-4b38-a10b-98bb7a885446 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:02.650961 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f44e9-e709-4b38-a10b-98bb7a885446 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:02.651259 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f44e9-e709-4b38-a10b-98bb7a885446 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:02.652679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f44e9-e709-4b38-a10b-98bb7a885446 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:02.665547 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 622/1211] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:02.671389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2241b19-ca5a-47d8-8894-7b6067e0af1c None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:02.775287 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2241b19-ca5a-47d8-8894-7b6067e0af1c None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:02.776796 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:02.817303 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b015c6b-1630-4ac9-8087-31b87e542d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:02.817940 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b015c6b-1630-4ac9-8087-31b87e542d89 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:02.818215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b015c6b-1630-4ac9-8087-31b87e542d89 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:02.818482 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b015c6b-1630-4ac9-8087-31b87e542d89 None admin] PATH_INFO: `/v3/users/placement` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:02.826264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b015c6b-1630-4ac9-8087-31b87e542d89 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:02.827700 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b015c6b-1630-4ac9-8087-31b87e542d89 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:02.839992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-6b015c6b-1630-4ac9-8087-31b87e542d89 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:28:02.839992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:02.839992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:02.839992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:02.839992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:02.839992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:02.839992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:02.839992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:02.839992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:02.839992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:02.839992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:02.839992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:02.839992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:28:02.839992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:28:02.839992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:28:02.839992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:28:02.841256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:02.841256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:02.841256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:28:02.841256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:28:02.841256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:28:02.841256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:28:02.841256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:02.841256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:02.841256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:02.841256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:02.841256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:02.841256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:02.841256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:02.841256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:02.841256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:02.841256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:02.841256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:02.841256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:02.842443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:02.842443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:02.842443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:28:02.842443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:28:02.842443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:28:02.842443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:28:02.842443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:28:02.842443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:28:02.842443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:28:02.842443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:02.844463 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 590/1212] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:02.847763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f44e9-e709-4b38-a10b-98bb7a885446 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:02.953120 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f44e9-e709-4b38-a10b-98bb7a885446 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:02.989377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1d544-8c33-4845-b8b4-d457150d1f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:02.989988 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1d544-8c33-4845-b8b4-d457150d1f4f None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:02.990293 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1d544-8c33-4845-b8b4-d457150d1f4f None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:02.990563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1d544-8c33-4845-b8b4-d457150d1f4f None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:02.991291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1d544-8c33-4845-b8b4-d457150d1f4f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:02.991591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1d544-8c33-4845-b8b4-d457150d1f4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:02.993046 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1d544-8c33-4845-b8b4-d457150d1f4f None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:03.010721 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 623/1213] 10.222.0.59 () {56 vars in 1143 bytes} [Mon Aug 29 22:28:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:03.016698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b015c6b-1630-4ac9-8087-31b87e542d89 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:03.120966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b015c6b-1630-4ac9-8087-31b87e542d89 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:03.157799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e6c44-bdd3-4b38-9cc4-4d66fd2d713f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:03.158420 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e6c44-bdd3-4b38-9cc4-4d66fd2d713f None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:03.158695 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e6c44-bdd3-4b38-9cc4-4d66fd2d713f None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:03.158965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e6c44-bdd3-4b38-9cc4-4d66fd2d713f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:03.191246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:03.230994 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e6c44-bdd3-4b38-9cc4-4d66fd2d713f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:03.232578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e6c44-bdd3-4b38-9cc4-4d66fd2d713f None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:03.336657 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 591/1214] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:28:03.342625 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1d544-8c33-4845-b8b4-d457150d1f4f None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:03.362376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:03.451979 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1d544-8c33-4845-b8b4-d457150d1f4f None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:03.489001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519c1eb-9e1f-471f-9fc8-fa6b54c1afc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:03.489641 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519c1eb-9e1f-471f-9fc8-fa6b54c1afc2 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:03.489930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519c1eb-9e1f-471f-9fc8-fa6b54c1afc2 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:03.490233 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519c1eb-9e1f-471f-9fc8-fa6b54c1afc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:03.500654 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2519c1eb-9e1f-471f-9fc8-fa6b54c1afc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:03.502184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2519c1eb-9e1f-471f-9fc8-fa6b54c1afc2 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:03.517714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-2519c1eb-9e1f-471f-9fc8-fa6b54c1afc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:03.517714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:03.517714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:03.517714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:03.517714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:03.517714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:03.517714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:03.517714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:03.517714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:03.517714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:03.517714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:03.517714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:03.517714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:28:03.517714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:28:03.517714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:28:03.517714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:28:03.519806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:03.519806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:03.519806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:03.519806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:03.519806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:03.519806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:03.519806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:03.519806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:03.519806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:03.519806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:03.519806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:03.519806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:03.519806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:03.519806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:03.519806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:03.519806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:03.519806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:28:03.520958 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:28:03.520958 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:03.520958 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:28:03.520958 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 624/1215] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:03.525652 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e6c44-bdd3-4b38-9cc4-4d66fd2d713f None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:03.589170 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:03.628824 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e6c44-bdd3-4b38-9cc4-4d66fd2d713f None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:03.665820 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75c720-e916-40e0-a87b-e339e3821e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:03.666451 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75c720-e916-40e0-a87b-e339e3821e74 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:03.666731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75c720-e916-40e0-a87b-e339e3821e74 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:03.667002 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75c720-e916-40e0-a87b-e339e3821e74 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:03.667674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75c720-e916-40e0-a87b-e339e3821e74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:03.667976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75c720-e916-40e0-a87b-e339e3821e74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:03.669385 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75c720-e916-40e0-a87b-e339e3821e74 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:03.681859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 592/1216] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:03.688049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519c1eb-9e1f-471f-9fc8-fa6b54c1afc2 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:03.760846 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:03.797849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519c1eb-9e1f-471f-9fc8-fa6b54c1afc2 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:03.832339 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab15ea-ecf7-4d11-87db-a98145a42039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:03.832956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab15ea-ecf7-4d11-87db-a98145a42039 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:03.833227 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab15ea-ecf7-4d11-87db-a98145a42039 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:03.833489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab15ea-ecf7-4d11-87db-a98145a42039 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:03.839861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab15ea-ecf7-4d11-87db-a98145a42039 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:03.841444 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab15ea-ecf7-4d11-87db-a98145a42039 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:03.852381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-caab15ea-ecf7-4d11-87db-a98145a42039 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:28:03.852381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:03.852381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:03.852381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:03.852381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:03.852381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:03.852381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:03.852381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:03.852381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:03.852381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:03.852381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:03.852381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:03.852381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:28:03.852381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:28:03.852381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:28:03.855311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:28:03.855311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:03.855311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:03.855311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:03.855311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:03.855311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:03.855311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:03.855311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:03.855311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:03.855311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:03.855311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:03.855311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:03.855311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:03.855311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:03.855311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:03.855311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:03.855311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:03.856420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:28:03.856420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:28:03.856420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:28:03.856420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:28:03.856420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:28:03.856420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:28:03.856420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:28:03.856420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:28:03.856420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 625/1217] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:28:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:28:03.859976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75c720-e916-40e0-a87b-e339e3821e74 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:03.962172 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75c720-e916-40e0-a87b-e339e3821e74 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:03.985801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:03.997663 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f220cc-7bd5-42a1-a4e5-6d88a4b37c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:03.998268 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f220cc-7bd5-42a1-a4e5-6d88a4b37c50 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:03.998543 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f220cc-7bd5-42a1-a4e5-6d88a4b37c50 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:03.998814 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f220cc-7bd5-42a1-a4e5-6d88a4b37c50 None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:03.999499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f220cc-7bd5-42a1-a4e5-6d88a4b37c50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:03.999792 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f220cc-7bd5-42a1-a4e5-6d88a4b37c50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:04.001267 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f220cc-7bd5-42a1-a4e5-6d88a4b37c50 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:04.013553 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 593/1218] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:28:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:04.020303 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab15ea-ecf7-4d11-87db-a98145a42039 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:04.120406 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab15ea-ecf7-4d11-87db-a98145a42039 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:04.149247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:04.155090 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1891ca6-73c2-41ee-b18b-4db4868e09b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:04.155683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1891ca6-73c2-41ee-b18b-4db4868e09b5 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:04.155955 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1891ca6-73c2-41ee-b18b-4db4868e09b5 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:04.156244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1891ca6-73c2-41ee-b18b-4db4868e09b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:04.156954 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1891ca6-73c2-41ee-b18b-4db4868e09b5 None admin] RBAC: Adding query filter params (role.id=f91580fb2e5840bcb4f4054732e2e796, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=7215ee41033742d79671f4f42b81cd06) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:04.157251 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1891ca6-73c2-41ee-b18b-4db4868e09b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:04.158659 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1891ca6-73c2-41ee-b18b-4db4868e09b5 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:04.164218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 626/1219] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:04 2022] GET /identity/v3/role_assignments?user.id=7215ee41033742d79671f4f42b81cd06&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=f91580fb2e5840bcb4f4054732e2e796 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:06.078075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19283358-7849-4a07-b508-bde5a9da07bf None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:06.078616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19283358-7849-4a07-b508-bde5a9da07bf None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:06.078727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19283358-7849-4a07-b508-bde5a9da07bf None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:06.079988 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 594/1220] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:06.085402 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662afc6-db52-49e7-9990-164555869923 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:06.085837 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662afc6-db52-49e7-9990-164555869923 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:06.085936 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662afc6-db52-49e7-9990-164555869923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:06.104105 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-c662afc6-db52-49e7-9990-164555869923 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:06.105327 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c662afc6-db52-49e7-9990-164555869923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:06.171766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 627/1221] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:28:06.443516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19283358-7849-4a07-b508-bde5a9da07bf None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:06.544734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19283358-7849-4a07-b508-bde5a9da07bf None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:06.578525 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4541be3-3696-4181-85a5-a41fe4d41464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:06.579140 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4541be3-3696-4181-85a5-a41fe4d41464 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:06.579409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4541be3-3696-4181-85a5-a41fe4d41464 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:06.579669 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4541be3-3696-4181-85a5-a41fe4d41464 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:06.589698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4541be3-3696-4181-85a5-a41fe4d41464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:06.594524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-e4541be3-3696-4181-85a5-a41fe4d41464 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:28:06.594524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:06.594524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:06.594524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:06.594524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:06.594524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:06.594524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:06.594524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:06.594524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:06.594524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:06.594524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:06.594524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:06.594524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:06.594524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:06.594524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:06.594524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:06.595727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:06.595727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:06.595727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:06.595727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:06.595727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:06.595727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:06.595727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:06.595727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:06.595727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:06.595727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:06.595727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:06.595727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:06.595727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:06.595727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:06.595727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:06.595727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:06.596733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:06.596733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:06.596733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:06.596733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:06.596733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:06.596733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:06.596733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:06.596733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:06.596733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:06.596733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:06.596733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:28:06.596733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:06.596733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 595/1222] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:28:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:06.602444 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662afc6-db52-49e7-9990-164555869923 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:06.701859 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662afc6-db52-49e7-9990-164555869923 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:06.725320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:06.736267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddfac0c-e6af-4b9c-a487-e5a61994087b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:06.736859 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddfac0c-e6af-4b9c-a487-e5a61994087b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:06.737132 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddfac0c-e6af-4b9c-a487-e5a61994087b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:06.737397 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddfac0c-e6af-4b9c-a487-e5a61994087b None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:06.738038 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddfac0c-e6af-4b9c-a487-e5a61994087b None admin] RBAC: Adding query filter params (name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:06.738329 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddfac0c-e6af-4b9c-a487-e5a61994087b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:06.740178 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddfac0c-e6af-4b9c-a487-e5a61994087b None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:06.747990 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 628/1223] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:28:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:06.754031 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4541be3-3696-4181-85a5-a41fe4d41464 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:06.799798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:06.855916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4541be3-3696-4181-85a5-a41fe4d41464 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:06.890471 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6882ae-a664-4995-aefb-f2e7fae5e954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:06.891121 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6882ae-a664-4995-aefb-f2e7fae5e954 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:06.891396 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6882ae-a664-4995-aefb-f2e7fae5e954 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:06.891666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6882ae-a664-4995-aefb-f2e7fae5e954 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:06.901678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6882ae-a664-4995-aefb-f2e7fae5e954 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:06.903585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6882ae-a664-4995-aefb-f2e7fae5e954 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:06.917923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-3f6882ae-a664-4995-aefb-f2e7fae5e954 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:06.917923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:06.917923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:06.917923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:06.917923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:06.917923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:06.917923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:06.917923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:06.917923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:06.917923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:06.917923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:06.917923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:06.917923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:28:06.917923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:28:06.917923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:28:06.917923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:28:06.918903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:06.918903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:06.918903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:06.918903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:06.918903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:06.918903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:06.918903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:06.918903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:06.918903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:06.918903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:06.918903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:06.918903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:06.918903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:06.918903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:06.918903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:06.918903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:06.918903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:28:06.920419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:28:06.920419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:06.920419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:06.920419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 596/1224] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:06.925624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddfac0c-e6af-4b9c-a487-e5a61994087b None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:07.029147 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddfac0c-e6af-4b9c-a487-e5a61994087b None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:07.062158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d819ff8-c904-48f6-be44-3d4c2ff1c153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:07.062797 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d819ff8-c904-48f6-be44-3d4c2ff1c153 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:07.063071 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d819ff8-c904-48f6-be44-3d4c2ff1c153 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:07.063342 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d819ff8-c904-48f6-be44-3d4c2ff1c153 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:07.063969 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d819ff8-c904-48f6-be44-3d4c2ff1c153 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:07.064329 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d819ff8-c904-48f6-be44-3d4c2ff1c153 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:07.065692 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d819ff8-c904-48f6-be44-3d4c2ff1c153 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:07.078586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 629/1225] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:07.084653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6882ae-a664-4995-aefb-f2e7fae5e954 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:07.179945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:07.186701 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6882ae-a664-4995-aefb-f2e7fae5e954 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:07.219764 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9bb47-1814-454a-b9af-d4931d15b757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:07.220413 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9bb47-1814-454a-b9af-d4931d15b757 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:07.220688 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9bb47-1814-454a-b9af-d4931d15b757 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:07.220954 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9bb47-1814-454a-b9af-d4931d15b757 None admin] PATH_INFO: `/v3/users/placement` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:07.228969 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9bb47-1814-454a-b9af-d4931d15b757 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:07.230400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9bb47-1814-454a-b9af-d4931d15b757 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:07.242269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-43b9bb47-1814-454a-b9af-d4931d15b757 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:28:07.242269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:07.242269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:07.242269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:07.242269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:07.242269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:07.242269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:07.242269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:07.242269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:07.242269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:07.242269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:07.242269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:07.242269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:28:07.242269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:28:07.242269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:28:07.242269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:28:07.246677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:07.246677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:07.246677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:28:07.246677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:28:07.246677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:28:07.246677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:28:07.246677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:07.246677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:07.246677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:07.246677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:07.246677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:07.246677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:07.246677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:07.246677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:07.246677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:07.246677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:07.246677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:07.246677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:07.247751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:07.247751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:07.247751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:28:07.247751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:28:07.247751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:28:07.247751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:28:07.247751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:28:07.247751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:28:07.247751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:28:07.247751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:07.247751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 597/1226] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:07.250350 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d819ff8-c904-48f6-be44-3d4c2ff1c153 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:07.299987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:07.353100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d819ff8-c904-48f6-be44-3d4c2ff1c153 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:07.387474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018629ae-63c2-471b-8f26-1c0fdc0588b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:07.388084 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018629ae-63c2-471b-8f26-1c0fdc0588b0 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:07.388385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018629ae-63c2-471b-8f26-1c0fdc0588b0 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:07.388651 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018629ae-63c2-471b-8f26-1c0fdc0588b0 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:07.389375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018629ae-63c2-471b-8f26-1c0fdc0588b0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:07.389672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018629ae-63c2-471b-8f26-1c0fdc0588b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:07.391134 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018629ae-63c2-471b-8f26-1c0fdc0588b0 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:07.410098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 630/1227] 10.222.0.59 () {56 vars in 1143 bytes} [Mon Aug 29 22:28:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:07.416266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9bb47-1814-454a-b9af-d4931d15b757 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:07.520662 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9bb47-1814-454a-b9af-d4931d15b757 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:07.555697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f59d8-324a-4157-a24a-d66bfe1b434c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:07.556264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f59d8-324a-4157-a24a-d66bfe1b434c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:07.556541 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f59d8-324a-4157-a24a-d66bfe1b434c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:07.556809 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f59d8-324a-4157-a24a-d66bfe1b434c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:07.567513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f59d8-324a-4157-a24a-d66bfe1b434c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:07.613957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f59d8-324a-4157-a24a-d66bfe1b434c None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:07.613957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-892f59d8-324a-4157-a24a-d66bfe1b434c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:07.613957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:07.613957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:07.613957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:07.613957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:07.613957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:07.613957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:07.613957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:07.613957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:07.613957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:07.613957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:07.613957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:07.613957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:28:07.614870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:28:07.614870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:28:07.614870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:28:07.614870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:07.614870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:07.614870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:07.614870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:07.614870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:07.614870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:07.614870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:07.614870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:07.614870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:07.614870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:07.614870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:07.614870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:07.614870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:07.614870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:07.615963 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:07.615963 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:07.615963 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:28:07.615963 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:28:07.615963 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:07.615963 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:07.615963 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 598/1228] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:07.616454 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018629ae-63c2-471b-8f26-1c0fdc0588b0 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:07.695390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018629ae-63c2-471b-8f26-1c0fdc0588b0 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:07.697747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:07.730561 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f0c26-7ed4-4551-9321-00b7db029cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:07.731168 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f0c26-7ed4-4551-9321-00b7db029cd6 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:07.731438 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f0c26-7ed4-4551-9321-00b7db029cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:07.731707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f0c26-7ed4-4551-9321-00b7db029cd6 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:07.732371 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f0c26-7ed4-4551-9321-00b7db029cd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:07.732668 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f0c26-7ed4-4551-9321-00b7db029cd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:07.734021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f0c26-7ed4-4551-9321-00b7db029cd6 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:07.745911 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 631/1229] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:07.752102 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f59d8-324a-4157-a24a-d66bfe1b434c None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:07.761648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:07.854613 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f59d8-324a-4157-a24a-d66bfe1b434c None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:07.889598 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98602f32-ff41-4b52-8c17-b60f3fe6814c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:07.890233 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98602f32-ff41-4b52-8c17-b60f3fe6814c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:07.890510 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98602f32-ff41-4b52-8c17-b60f3fe6814c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:07.890781 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98602f32-ff41-4b52-8c17-b60f3fe6814c None admin] PATH_INFO: `/v3/projects/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:07.898115 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98602f32-ff41-4b52-8c17-b60f3fe6814c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:07.899597 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98602f32-ff41-4b52-8c17-b60f3fe6814c None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:07.914371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-98602f32-ff41-4b52-8c17-b60f3fe6814c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:28:07.914371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:07.914371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:07.914371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:07.914371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:07.914371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:07.914371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:07.914371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:07.914371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:07.914371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:07.914371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:07.914371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:07.914371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:28:07.914371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:28:07.914371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:28:07.916731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:28:07.916731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:07.916731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:07.916731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:07.916731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:07.916731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:07.916731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:07.916731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:07.916731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:07.916731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:07.916731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:07.916731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:07.916731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:07.916731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:07.916731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:07.916731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:07.916731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:07.917632 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:28:07.917632 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:28:07.917632 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:28:07.917632 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:28:07.917632 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:28:07.917632 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:28:07.917632 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:28:07.917632 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:07.917632 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 599/1230] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:28:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:07.921493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f0c26-7ed4-4551-9321-00b7db029cd6 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:08.039099 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f0c26-7ed4-4551-9321-00b7db029cd6 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:08.072971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda2f57-8943-4fc5-b24f-ebb1a0092241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:08.073625 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda2f57-8943-4fc5-b24f-ebb1a0092241 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:08.074069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda2f57-8943-4fc5-b24f-ebb1a0092241 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:08.074216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda2f57-8943-4fc5-b24f-ebb1a0092241 None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:08.074905 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bda2f57-8943-4fc5-b24f-ebb1a0092241 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:08.075205 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bda2f57-8943-4fc5-b24f-ebb1a0092241 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:08.076692 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bda2f57-8943-4fc5-b24f-ebb1a0092241 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:08.089275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 632/1231] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:28:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:08.095430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98602f32-ff41-4b52-8c17-b60f3fe6814c None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:08.161221 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:08.195489 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98602f32-ff41-4b52-8c17-b60f3fe6814c None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:08.230119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd46e2d-8c36-44c9-8475-c5c0bb719fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:08.230827 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd46e2d-8c36-44c9-8475-c5c0bb719fa2 None admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:08.231117 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd46e2d-8c36-44c9-8475-c5c0bb719fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:08.231394 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd46e2d-8c36-44c9-8475-c5c0bb719fa2 None admin] PATH_INFO: `/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/7215ee41033742d79671f4f42b81cd06/roles/f91580fb2e5840bcb4f4054732e2e796` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:08.246386 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd46e2d-8c36-44c9-8475-c5c0bb719fa2 None admin] RBAC: Authorizing `identity:create_grant(project_id=7799820fc7c64fd6a99a0a9ccf1338e3, user_id=7215ee41033742d79671f4f42b81cd06, role_id=f91580fb2e5840bcb4f4054732e2e796)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:08.247959 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd46e2d-8c36-44c9-8475-c5c0bb719fa2 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:08.369305 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 600/1232] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:28:08 2022] PUT /identity/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/7215ee41033742d79671f4f42b81cd06/roles/f91580fb2e5840bcb4f4054732e2e796 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:28:10.350417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffbc48-cd84-45c9-98cf-807597f5e0e2 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:10.351055 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffbc48-cd84-45c9-98cf-807597f5e0e2 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:10.351055 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffbc48-cd84-45c9-98cf-807597f5e0e2 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:10.352278 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 633/1233] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:28:10.360072 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea83990-1fe6-4998-8887-4b453be50f34 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:10.360672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea83990-1fe6-4998-8887-4b453be50f34 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:10.360866 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea83990-1fe6-4998-8887-4b453be50f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:10.381072 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-9ea83990-1fe6-4998-8887-4b453be50f34 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:10.382293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-9ea83990-1fe6-4998-8887-4b453be50f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:10.455508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 601/1234] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 22:28:10.737795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffbc48-cd84-45c9-98cf-807597f5e0e2 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:10.764518 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:10.848894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffbc48-cd84-45c9-98cf-807597f5e0e2 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:10.885140 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb30e4-af65-42be-aaa7-0d46ecc6cd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:10.885761 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb30e4-af65-42be-aaa7-0d46ecc6cd35 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:10.886036 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb30e4-af65-42be-aaa7-0d46ecc6cd35 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:10.886307 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb30e4-af65-42be-aaa7-0d46ecc6cd35 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:10.897365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb30e4-af65-42be-aaa7-0d46ecc6cd35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:10.902236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-57eb30e4-af65-42be-aaa7-0d46ecc6cd35 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:28:10.902236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:10.902236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:10.902236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:10.902236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:10.902236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:10.902236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:10.902236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:10.902236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:10.902236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:10.902236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:10.902236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:10.902236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:10.902236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:10.902236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:10.902236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:10.903375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:10.903375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:10.903375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:10.903375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:10.903375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:10.903375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:10.903375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:10.903375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:10.903375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:10.903375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:10.903375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:10.903375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:10.903375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:10.903375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:10.903375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:10.903375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:10.904533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:10.904533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:10.904533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:10.904533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:10.904533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:10.904533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:10.904533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:10.904533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:10.904533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:10.904533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:10.904533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 22:28:10.904533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:28:10.904533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 634/1235] 10.222.0.59 () {56 vars in 1102 bytes} [Mon Aug 29 22:28:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:10.909748 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea83990-1fe6-4998-8887-4b453be50f34 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:10.959671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:11.014883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea83990-1fe6-4998-8887-4b453be50f34 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:11.050433 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d92418-2d38-493b-83d5-45601bb6dd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:11.051141 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d92418-2d38-493b-83d5-45601bb6dd9a None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:11.051635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d92418-2d38-493b-83d5-45601bb6dd9a None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:11.052024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d92418-2d38-493b-83d5-45601bb6dd9a None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:11.052723 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d92418-2d38-493b-83d5-45601bb6dd9a None admin] RBAC: Adding query filter params (name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:11.053023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d92418-2d38-493b-83d5-45601bb6dd9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:11.054434 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d92418-2d38-493b-83d5-45601bb6dd9a None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:11.064312 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 602/1236] 10.222.0.59 () {56 vars in 1103 bytes} [Mon Aug 29 22:28:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:11.071014 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb30e4-af65-42be-aaa7-0d46ecc6cd35 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:11.158892 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:11.174850 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb30e4-af65-42be-aaa7-0d46ecc6cd35 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:11.211440 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe0b0d-1e95-47fb-847d-f25a7facc259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:11.212066 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe0b0d-1e95-47fb-847d-f25a7facc259 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:11.212376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe0b0d-1e95-47fb-847d-f25a7facc259 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:11.212716 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe0b0d-1e95-47fb-847d-f25a7facc259 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:11.223427 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe0b0d-1e95-47fb-847d-f25a7facc259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:11.225079 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe0b0d-1e95-47fb-847d-f25a7facc259 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:11.240630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-37fe0b0d-1e95-47fb-847d-f25a7facc259 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:11.240630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:11.240630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:11.240630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:11.240630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:11.240630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:11.240630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:11.240630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:11.240630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:11.240630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:11.240630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:11.240630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:11.240630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:28:11.240630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:28:11.240630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:28:11.240630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:28:11.241868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:11.241868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:11.241868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:11.241868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:11.241868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:11.241868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:11.241868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:11.241868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:11.241868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:11.241868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:11.241868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:11.241868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:11.241868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:11.241868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:11.241868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:11.241868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:11.241868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:28:11.243098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:28:11.243098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:11.243098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:28:11.243098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 635/1237] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:11.248077 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d92418-2d38-493b-83d5-45601bb6dd9a None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:11.355475 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d92418-2d38-493b-83d5-45601bb6dd9a None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:11.366087 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:11.390922 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ed050-c9a9-45f9-88a8-63327764ff2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:11.391546 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ed050-c9a9-45f9-88a8-63327764ff2d None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:11.391803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ed050-c9a9-45f9-88a8-63327764ff2d None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:11.392070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ed050-c9a9-45f9-88a8-63327764ff2d None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:11.392745 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ed050-c9a9-45f9-88a8-63327764ff2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:11.393031 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ed050-c9a9-45f9-88a8-63327764ff2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:11.394387 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ed050-c9a9-45f9-88a8-63327764ff2d None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:11.406913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 603/1238] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:11.413359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe0b0d-1e95-47fb-847d-f25a7facc259 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:11.518704 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe0b0d-1e95-47fb-847d-f25a7facc259 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:11.554490 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22a23b-8a5c-4cc4-897a-b07f525063b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:11.555117 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22a23b-8a5c-4cc4-897a-b07f525063b6 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:11.555460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22a23b-8a5c-4cc4-897a-b07f525063b6 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:11.555735 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22a23b-8a5c-4cc4-897a-b07f525063b6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:11.558589 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:11.563869 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22a23b-8a5c-4cc4-897a-b07f525063b6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:11.565358 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22a23b-8a5c-4cc4-897a-b07f525063b6 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:11.577618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-1d22a23b-8a5c-4cc4-897a-b07f525063b6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:28:11.577618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:11.577618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:11.577618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:11.577618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:11.577618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:11.577618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:11.577618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:11.577618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:11.577618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:11.577618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:11.577618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:11.577618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:28:11.577618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:28:11.577618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:28:11.577618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:28:11.578825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:11.578825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:11.578825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:28:11.578825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:28:11.578825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:28:11.578825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:28:11.578825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:11.578825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:11.578825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:11.578825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:11.578825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:11.578825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:11.578825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:11.578825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:11.578825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:11.578825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:11.578825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:11.578825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:11.580095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:11.580095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:11.580095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:28:11.580095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:28:11.580095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:28:11.580095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:28:11.580095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:28:11.580095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:28:11.580095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:28:11.580095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:28:11.580095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 636/1239] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:11.584828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ed050-c9a9-45f9-88a8-63327764ff2d None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:11.693785 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ed050-c9a9-45f9-88a8-63327764ff2d None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:11.729646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe723f0-f0de-4622-b4c8-7e34d0b6f2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:11.730253 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe723f0-f0de-4622-b4c8-7e34d0b6f2d7 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:11.730534 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe723f0-f0de-4622-b4c8-7e34d0b6f2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:11.730803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe723f0-f0de-4622-b4c8-7e34d0b6f2d7 None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:11.731532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe723f0-f0de-4622-b4c8-7e34d0b6f2d7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:11.731830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe723f0-f0de-4622-b4c8-7e34d0b6f2d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:11.733290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe723f0-f0de-4622-b4c8-7e34d0b6f2d7 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:11.752298 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 604/1240] 10.222.0.59 () {56 vars in 1143 bytes} [Mon Aug 29 22:28:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:11.760931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22a23b-8a5c-4cc4-897a-b07f525063b6 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:11.866533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22a23b-8a5c-4cc4-897a-b07f525063b6 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:11.900777 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bcb01-1e99-499c-94c6-b02137b7c3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:11.901377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bcb01-1e99-499c-94c6-b02137b7c3b2 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:11.901760 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bcb01-1e99-499c-94c6-b02137b7c3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:11.902029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bcb01-1e99-499c-94c6-b02137b7c3b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:11.968127 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:11.975292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bcb01-1e99-499c-94c6-b02137b7c3b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:11.976906 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bcb01-1e99-499c-94c6-b02137b7c3b2 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:12.080616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 637/1241] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:28:12.087328 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe723f0-f0de-4622-b4c8-7e34d0b6f2d7 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:12.117300 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:12.193205 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe723f0-f0de-4622-b4c8-7e34d0b6f2d7 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:12.231529 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2662a-1565-437d-8b78-65e552456f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:12.232244 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2662a-1565-437d-8b78-65e552456f75 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:12.232521 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2662a-1565-437d-8b78-65e552456f75 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:12.232792 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2662a-1565-437d-8b78-65e552456f75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:12.244818 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2662a-1565-437d-8b78-65e552456f75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:12.246254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2662a-1565-437d-8b78-65e552456f75 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:12.262083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-83f2662a-1565-437d-8b78-65e552456f75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:12.262083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:12.262083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:12.262083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:12.262083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:12.262083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:12.262083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:12.262083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:12.262083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:12.262083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:12.262083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:12.262083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:12.262083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:28:12.262083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:28:12.262083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:28:12.262083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:28:12.263319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:12.263319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:12.263319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:12.263319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:12.263319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:12.263319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:12.263319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:12.263319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:12.263319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:12.263319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:12.263319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:12.263319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:12.263319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:12.263319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:12.263319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:12.263319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:12.263319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:28:12.264484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:28:12.264484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:12.264484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:12.264484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 605/1242] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:12.269152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bcb01-1e99-499c-94c6-b02137b7c3b2 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:12.372792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:12.378508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bcb01-1e99-499c-94c6-b02137b7c3b2 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:12.414407 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38b159-3ca0-4e4e-af2c-049c14942bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:12.415067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38b159-3ca0-4e4e-af2c-049c14942bb2 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:12.415258 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38b159-3ca0-4e4e-af2c-049c14942bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:12.415522 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38b159-3ca0-4e4e-af2c-049c14942bb2 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:12.416181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38b159-3ca0-4e4e-af2c-049c14942bb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:12.416483 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38b159-3ca0-4e4e-af2c-049c14942bb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:12.417839 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38b159-3ca0-4e4e-af2c-049c14942bb2 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:12.430446 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 638/1243] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:12.436377 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2662a-1565-437d-8b78-65e552456f75 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:12.518938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:12.541310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2662a-1565-437d-8b78-65e552456f75 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:12.577379 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53002559-8b61-480b-b4c3-a557095a393b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:12.578010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53002559-8b61-480b-b4c3-a557095a393b None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:12.578286 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53002559-8b61-480b-b4c3-a557095a393b None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:12.578689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53002559-8b61-480b-b4c3-a557095a393b None admin] PATH_INFO: `/v3/projects/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:12.587205 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53002559-8b61-480b-b4c3-a557095a393b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:12.588691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53002559-8b61-480b-b4c3-a557095a393b None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:12.601095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-53002559-8b61-480b-b4c3-a557095a393b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:28:12.601095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:12.601095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:12.601095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:12.601095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:12.601095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:12.601095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:12.601095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:12.601095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:12.601095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:12.601095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:12.601095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:12.601095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:28:12.601095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:28:12.601095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:28:12.602246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:28:12.602246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:12.602246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:12.602246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:12.602246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:12.602246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:12.602246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:12.602246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:12.602246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:12.602246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:12.602246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:12.602246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:12.602246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:12.602246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:12.602246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:12.602246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:12.602246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:12.603357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:28:12.603357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:28:12.603357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:28:12.603357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:28:12.603357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:28:12.603357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:28:12.603357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:28:12.603357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:12.603357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 606/1244] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:28:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:12.608344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38b159-3ca0-4e4e-af2c-049c14942bb2 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:12.716428 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38b159-3ca0-4e4e-af2c-049c14942bb2 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:12.752366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3e64e-2d5c-4e2b-b0ba-8915df7f6222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:12.752963 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3e64e-2d5c-4e2b-b0ba-8915df7f6222 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:12.753239 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3e64e-2d5c-4e2b-b0ba-8915df7f6222 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:12.753509 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3e64e-2d5c-4e2b-b0ba-8915df7f6222 None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:12.754441 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3e64e-2d5c-4e2b-b0ba-8915df7f6222 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:12.754957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3e64e-2d5c-4e2b-b0ba-8915df7f6222 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:12.756416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3e64e-2d5c-4e2b-b0ba-8915df7f6222 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:12.768643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 639/1245] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:28:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:12.776044 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53002559-8b61-480b-b4c3-a557095a393b None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:12.892363 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53002559-8b61-480b-b4c3-a557095a393b None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:12.931530 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a71e3e-0fcd-4bc6-a535-10caec38f675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:12.932161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a71e3e-0fcd-4bc6-a535-10caec38f675 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:12.932441 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a71e3e-0fcd-4bc6-a535-10caec38f675 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:12.932711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a71e3e-0fcd-4bc6-a535-10caec38f675 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:12.933448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a71e3e-0fcd-4bc6-a535-10caec38f675 None admin] RBAC: Adding query filter params (role.id=f91580fb2e5840bcb4f4054732e2e796, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=7215ee41033742d79671f4f42b81cd06) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:12.933751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a71e3e-0fcd-4bc6-a535-10caec38f675 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:12.935164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a71e3e-0fcd-4bc6-a535-10caec38f675 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:12.942696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 607/1246] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:12 2022] GET /identity/v3/role_assignments?user.id=7215ee41033742d79671f4f42b81cd06&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=f91580fb2e5840bcb4f4054732e2e796 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:14.957924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9744b2-fe40-450a-b0cb-f2750cb94733 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:14.958527 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9744b2-fe40-450a-b0cb-f2750cb94733 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:14.958527 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9744b2-fe40-450a-b0cb-f2750cb94733 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:14.959895 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 640/1247] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:14.965432 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084eed9a-2a07-42ec-9608-b532cd7c6fa7 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:14.965864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084eed9a-2a07-42ec-9608-b532cd7c6fa7 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:14.965965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084eed9a-2a07-42ec-9608-b532cd7c6fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:14.972316 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:14.985088 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-084eed9a-2a07-42ec-9608-b532cd7c6fa7 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:14.986261 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-084eed9a-2a07-42ec-9608-b532cd7c6fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:15.054515 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 608/1248] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:28:15.324533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9744b2-fe40-450a-b0cb-f2750cb94733 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:15.342366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:15.428267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9744b2-fe40-450a-b0cb-f2750cb94733 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:15.461802 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0bb858-8aed-4bde-8d82-806533f10458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:15.462420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bb858-8aed-4bde-8d82-806533f10458 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:15.462702 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bb858-8aed-4bde-8d82-806533f10458 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:15.462969 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bb858-8aed-4bde-8d82-806533f10458 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:15.473523 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0bb858-8aed-4bde-8d82-806533f10458 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:15.478287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-5c0bb858-8aed-4bde-8d82-806533f10458 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:15.478287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:15.478287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:15.478287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:15.478287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:15.478287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:15.478287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:15.478287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:15.478287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:15.478287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:15.478287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:15.478287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:15.478287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:15.478287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:15.478287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:15.478287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:15.479320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:15.479320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:15.479320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:15.479320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:15.479320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:15.479320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:15.479320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:15.479320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:15.479320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:15.479320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:15.479320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:15.479320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:15.479320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:15.479320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:15.479320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:15.479320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:15.480240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:15.480240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:15.480240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:15.480240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:15.480240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:15.480240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:15.480240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:15.480240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:15.480240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:15.480240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:15.480240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:15.480240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:28:15.480240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 641/1249] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:28:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:15.485617 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084eed9a-2a07-42ec-9608-b532cd7c6fa7 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:15.583986 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084eed9a-2a07-42ec-9608-b532cd7c6fa7 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:15.614811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:15.620399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3eed2a-b1ed-4f57-b05c-eda024ed3242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:15.621000 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3eed2a-b1ed-4f57-b05c-eda024ed3242 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:15.621275 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3eed2a-b1ed-4f57-b05c-eda024ed3242 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:15.621633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3eed2a-b1ed-4f57-b05c-eda024ed3242 None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:15.622286 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3eed2a-b1ed-4f57-b05c-eda024ed3242 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:15.622589 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3eed2a-b1ed-4f57-b05c-eda024ed3242 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:15.623991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3eed2a-b1ed-4f57-b05c-eda024ed3242 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:15.631984 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 609/1250] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:15.637956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0bb858-8aed-4bde-8d82-806533f10458 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:15.715764 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:15.739162 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0bb858-8aed-4bde-8d82-806533f10458 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:15.783344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3de75-e54b-4c94-842f-fdf57eae5e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:15.783961 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3de75-e54b-4c94-842f-fdf57eae5e95 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:15.784257 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3de75-e54b-4c94-842f-fdf57eae5e95 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:15.784522 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3de75-e54b-4c94-842f-fdf57eae5e95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:15.795066 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3de75-e54b-4c94-842f-fdf57eae5e95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:15.796515 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3de75-e54b-4c94-842f-fdf57eae5e95 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:15.811596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-19a3de75-e54b-4c94-842f-fdf57eae5e95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:15.811596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:15.811596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:15.811596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:15.811596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:15.811596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:15.811596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:15.811596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:15.811596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:15.811596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:15.811596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:15.811596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:15.811596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:28:15.811596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:28:15.811596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:28:15.811596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:28:15.818630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:15.818630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:15.818630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:15.818630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:15.818630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:15.818630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:15.818630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:15.818630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:15.818630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:15.818630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:15.818630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:15.818630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:15.818630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:15.818630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:15.818630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:15.818630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:15.818630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:28:15.819805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:28:15.819805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:15.819805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:28:15.819805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 642/1251] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:15.820094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3eed2a-b1ed-4f57-b05c-eda024ed3242 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:15.931849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3eed2a-b1ed-4f57-b05c-eda024ed3242 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:15.965720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe81e82-d11d-4e59-b338-020dea4dffda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:15.967476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe81e82-d11d-4e59-b338-020dea4dffda None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:15.967476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe81e82-d11d-4e59-b338-020dea4dffda None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:15.967476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe81e82-d11d-4e59-b338-020dea4dffda None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:15.967476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe81e82-d11d-4e59-b338-020dea4dffda None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:15.967948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe81e82-d11d-4e59-b338-020dea4dffda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:15.969107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe81e82-d11d-4e59-b338-020dea4dffda None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:15.982910 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 610/1252] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:15.990707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3de75-e54b-4c94-842f-fdf57eae5e95 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:16.098510 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3de75-e54b-4c94-842f-fdf57eae5e95 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:16.126082 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:16.134297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17285d3f-72b5-40ea-a9ef-f81f582a839a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:16.135002 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17285d3f-72b5-40ea-a9ef-f81f582a839a None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:16.135202 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17285d3f-72b5-40ea-a9ef-f81f582a839a None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:16.135473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17285d3f-72b5-40ea-a9ef-f81f582a839a None admin] PATH_INFO: `/v3/users/placement` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:16.143356 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17285d3f-72b5-40ea-a9ef-f81f582a839a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:16.145349 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17285d3f-72b5-40ea-a9ef-f81f582a839a None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:16.157575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-17285d3f-72b5-40ea-a9ef-f81f582a839a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:28:16.157575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:16.157575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:16.157575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:16.157575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:16.157575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:16.157575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:16.157575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:16.157575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:16.157575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:16.157575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:16.157575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:16.157575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:28:16.157575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:28:16.157575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:28:16.157575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:28:16.159827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:16.159827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:16.159827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:28:16.159827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:28:16.159827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:28:16.159827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:28:16.159827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:16.159827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:16.159827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:16.159827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:16.159827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:16.159827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:16.159827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:16.159827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:16.159827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:16.159827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:16.159827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:16.159827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:16.161389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:16.161389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:16.161389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:28:16.161389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:28:16.161389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:28:16.161389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:28:16.161389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:28:16.161389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:28:16.161389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:28:16.161389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:28:16.161389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 643/1253] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:16.165538 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe81e82-d11d-4e59-b338-020dea4dffda None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:16.225149 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:16.272814 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe81e82-d11d-4e59-b338-020dea4dffda None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:16.312092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e67be8-1ad7-47bb-8291-8bfca4356494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:16.312733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e67be8-1ad7-47bb-8291-8bfca4356494 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:16.313009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e67be8-1ad7-47bb-8291-8bfca4356494 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:16.313273 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e67be8-1ad7-47bb-8291-8bfca4356494 None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:16.314005 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e67be8-1ad7-47bb-8291-8bfca4356494 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:16.314591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e67be8-1ad7-47bb-8291-8bfca4356494 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:16.316123 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e67be8-1ad7-47bb-8291-8bfca4356494 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:16.339744 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 611/1254] 10.222.0.59 () {56 vars in 1143 bytes} [Mon Aug 29 22:28:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:16.345749 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17285d3f-72b5-40ea-a9ef-f81f582a839a None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:16.452913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17285d3f-72b5-40ea-a9ef-f81f582a839a None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:16.488177 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d44d00-a745-4ee9-af6e-428150e299cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:16.488673 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d44d00-a745-4ee9-af6e-428150e299cc None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:16.488945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d44d00-a745-4ee9-af6e-428150e299cc None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:16.489288 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d44d00-a745-4ee9-af6e-428150e299cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:16.552581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:16.565399 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d44d00-a745-4ee9-af6e-428150e299cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:16.567500 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d44d00-a745-4ee9-af6e-428150e299cc None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:16.679708 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 644/1255] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:28:16.685525 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e67be8-1ad7-47bb-8291-8bfca4356494 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:16.788354 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e67be8-1ad7-47bb-8291-8bfca4356494 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:16.802615 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:16.824882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e587669-df8b-401b-b2e5-d33d33c5899e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:16.825525 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e587669-df8b-401b-b2e5-d33d33c5899e None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:16.825789 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e587669-df8b-401b-b2e5-d33d33c5899e None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:16.826051 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e587669-df8b-401b-b2e5-d33d33c5899e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:16.837033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e587669-df8b-401b-b2e5-d33d33c5899e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:16.838509 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e587669-df8b-401b-b2e5-d33d33c5899e None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:16.853830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-1e587669-df8b-401b-b2e5-d33d33c5899e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:16.853830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:16.853830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:16.853830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:16.853830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:16.853830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:16.853830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:16.853830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:16.853830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:16.853830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:16.853830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:16.853830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:16.853830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:28:16.853830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:28:16.853830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:28:16.853830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:28:16.854856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:16.854856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:16.854856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:16.854856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:16.854856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:16.854856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:16.854856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:16.854856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:16.854856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:16.854856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:16.854856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:16.854856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:16.854856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:16.854856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:16.854856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:16.854856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:16.854856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:28:16.856281 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:28:16.856281 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:16.856281 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:16.856281 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 612/1256] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:16.866411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d44d00-a745-4ee9-af6e-428150e299cc None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:16.964708 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:16.977552 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d44d00-a745-4ee9-af6e-428150e299cc None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:17.011116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b610088-1f6e-45b3-9845-8994a82282bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:17.011782 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b610088-1f6e-45b3-9845-8994a82282bc None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:17.011967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b610088-1f6e-45b3-9845-8994a82282bc None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:17.012265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b610088-1f6e-45b3-9845-8994a82282bc None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:17.012892 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b610088-1f6e-45b3-9845-8994a82282bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:17.013185 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b610088-1f6e-45b3-9845-8994a82282bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:17.014535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b610088-1f6e-45b3-9845-8994a82282bc None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:17.026470 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 645/1257] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:17.032539 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e587669-df8b-401b-b2e5-d33d33c5899e None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:17.140396 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e587669-df8b-401b-b2e5-d33d33c5899e None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:17.175755 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078484c4-317c-41fa-8baf-c9419c336344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:17.176484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078484c4-317c-41fa-8baf-c9419c336344 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:17.176756 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078484c4-317c-41fa-8baf-c9419c336344 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:17.177023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078484c4-317c-41fa-8baf-c9419c336344 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:17.184122 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078484c4-317c-41fa-8baf-c9419c336344 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:17.185576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078484c4-317c-41fa-8baf-c9419c336344 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:17.196972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-078484c4-317c-41fa-8baf-c9419c336344 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:28:17.196972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:17.196972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:17.196972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:17.196972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:17.196972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:17.196972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:17.196972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:17.196972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:17.196972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:17.196972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:17.196972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:17.196972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:28:17.196972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:28:17.196972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:28:17.198024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:28:17.198024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:17.198024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:17.198024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:17.198024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:17.198024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:17.198024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:17.198024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:17.198024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:17.198024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:17.198024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:17.198024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:17.198024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:17.198024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:17.198024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:17.198024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:17.198024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:17.198916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:28:17.198916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:28:17.198916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:28:17.198916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:28:17.198916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:28:17.198916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:28:17.198916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:28:17.198916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:17.198916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 613/1258] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:28:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:17.204238 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b610088-1f6e-45b3-9845-8994a82282bc None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:17.304747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b610088-1f6e-45b3-9845-8994a82282bc None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:17.339645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60919c7b-f37a-46bd-beac-4654282a2118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:17.340269 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60919c7b-f37a-46bd-beac-4654282a2118 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:17.340546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60919c7b-f37a-46bd-beac-4654282a2118 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:17.340814 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60919c7b-f37a-46bd-beac-4654282a2118 None admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:17.341493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60919c7b-f37a-46bd-beac-4654282a2118 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:17.341801 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60919c7b-f37a-46bd-beac-4654282a2118 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:17.343211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60919c7b-f37a-46bd-beac-4654282a2118 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:17.355811 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 646/1259] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:28:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:17.362498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078484c4-317c-41fa-8baf-c9419c336344 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:17.388629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:17.469409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078484c4-317c-41fa-8baf-c9419c336344 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:17.509493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98c6a6-6332-4283-86cd-12d9bd6405a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:17.510112 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98c6a6-6332-4283-86cd-12d9bd6405a6 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:17.510388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98c6a6-6332-4283-86cd-12d9bd6405a6 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:17.510654 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98c6a6-6332-4283-86cd-12d9bd6405a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:17.511387 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98c6a6-6332-4283-86cd-12d9bd6405a6 None admin] RBAC: Adding query filter params (role.id=a836108447dc439e994611bc74d53d9c, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=7215ee41033742d79671f4f42b81cd06) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:17.511724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98c6a6-6332-4283-86cd-12d9bd6405a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:17.513422 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98c6a6-6332-4283-86cd-12d9bd6405a6 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:17.519821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 614/1260] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:17 2022] GET /identity/v3/role_assignments?user.id=7215ee41033742d79671f4f42b81cd06&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=a836108447dc439e994611bc74d53d9c => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:19.500977 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d704a-b11f-48ad-a376-7c1fcfc9f01a None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:19.501823 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d704a-b11f-48ad-a376-7c1fcfc9f01a None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:19.502034 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d704a-b11f-48ad-a376-7c1fcfc9f01a None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:19.503407 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 647/1261] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:28:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:19.509739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6c385-5042-4391-87e1-be712ce04a78 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:19.510412 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6c385-5042-4391-87e1-be712ce04a78 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:19.510602 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d6c385-5042-4391-87e1-be712ce04a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:19.529274 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-33d6c385-5042-4391-87e1-be712ce04a78 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:19.530487 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-33d6c385-5042-4391-87e1-be712ce04a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:19.603569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 615/1262] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:28:19.879322 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d704a-b11f-48ad-a376-7c1fcfc9f01a None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:19.890569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:19.986285 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d704a-b11f-48ad-a376-7c1fcfc9f01a None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:20.021942 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dd6ec-1a9d-41a2-b37a-86f9a280b50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:20.022556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dd6ec-1a9d-41a2-b37a-86f9a280b50b None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:20.022828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dd6ec-1a9d-41a2-b37a-86f9a280b50b None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:20.023093 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dd6ec-1a9d-41a2-b37a-86f9a280b50b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:20.033388 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746dd6ec-1a9d-41a2-b37a-86f9a280b50b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:20.038984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-746dd6ec-1a9d-41a2-b37a-86f9a280b50b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:20.038984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:20.038984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:20.038984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:20.038984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:20.038984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:20.038984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:20.038984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:20.038984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:20.038984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:20.038984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:20.038984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:20.038984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:20.038984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:20.038984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:20.038984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:20.048087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:20.048087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:20.048087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:20.048087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:20.048087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:20.048087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:20.048087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:20.048087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:20.048087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:20.048087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:20.048087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:20.048087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:20.048087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:20.048087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:20.048087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:20.048087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:20.049136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:20.049136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:20.049136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:20.049136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:20.049136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:20.049136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:20.049136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:20.049136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:20.049136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:20.049136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:20.049136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:20.049136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:28:20.049136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 648/1263] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:28:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:20.049974 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d6c385-5042-4391-87e1-be712ce04a78 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:20.065374 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:20.149647 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d6c385-5042-4391-87e1-be712ce04a78 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:20.188430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4483251-d8fc-40eb-a258-d8e220bef9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:20.191849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4483251-d8fc-40eb-a258-d8e220bef9fd None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:20.191849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4483251-d8fc-40eb-a258-d8e220bef9fd None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:20.191849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4483251-d8fc-40eb-a258-d8e220bef9fd None admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:20.191849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4483251-d8fc-40eb-a258-d8e220bef9fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:20.191849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4483251-d8fc-40eb-a258-d8e220bef9fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:20.192284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4483251-d8fc-40eb-a258-d8e220bef9fd None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:20.200487 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 616/1264] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:20.206283 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dd6ec-1a9d-41a2-b37a-86f9a280b50b None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:20.277477 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:20.313242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dd6ec-1a9d-41a2-b37a-86f9a280b50b None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:20.350014 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88db4b5-1216-4c1f-9f97-2fb80729771a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:20.350636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88db4b5-1216-4c1f-9f97-2fb80729771a None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:20.350914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88db4b5-1216-4c1f-9f97-2fb80729771a None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:20.351174 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88db4b5-1216-4c1f-9f97-2fb80729771a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:20.362505 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88db4b5-1216-4c1f-9f97-2fb80729771a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:20.364273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88db4b5-1216-4c1f-9f97-2fb80729771a None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:20.379838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-f88db4b5-1216-4c1f-9f97-2fb80729771a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:20.379838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:20.379838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:20.379838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:20.379838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:20.379838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:20.379838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:20.379838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:20.379838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:20.379838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:20.379838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:20.379838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:20.379838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:28:20.379838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:28:20.379838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:28:20.379838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:28:20.381213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:20.381213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:20.381213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:20.381213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:20.381213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:20.381213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:20.381213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:20.381213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:20.381213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:20.381213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:20.381213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:20.381213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:20.381213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:20.381213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:20.381213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:20.381213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:20.381213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:28:20.382623 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:28:20.382623 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:20.382623 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:28:20.382623 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 649/1265] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:20.387017 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4483251-d8fc-40eb-a258-d8e220bef9fd None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:20.475699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:20.494939 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4483251-d8fc-40eb-a258-d8e220bef9fd None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:20.529074 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275c7ac-ed78-471f-91f3-33eed59d981a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:20.529776 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275c7ac-ed78-471f-91f3-33eed59d981a None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:20.529931 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275c7ac-ed78-471f-91f3-33eed59d981a None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:20.530198 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275c7ac-ed78-471f-91f3-33eed59d981a None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:20.530832 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275c7ac-ed78-471f-91f3-33eed59d981a None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:20.531124 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275c7ac-ed78-471f-91f3-33eed59d981a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:20.532504 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275c7ac-ed78-471f-91f3-33eed59d981a None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:20.544375 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 617/1266] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:20.550319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88db4b5-1216-4c1f-9f97-2fb80729771a None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:20.657345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88db4b5-1216-4c1f-9f97-2fb80729771a None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:20.678426 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:20.693163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158c374-b559-4242-8637-5504e9effa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:20.694191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158c374-b559-4242-8637-5504e9effa4d None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:20.694683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158c374-b559-4242-8637-5504e9effa4d None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:20.695172 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158c374-b559-4242-8637-5504e9effa4d None admin] PATH_INFO: `/v3/users/placement` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:20.703889 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158c374-b559-4242-8637-5504e9effa4d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:20.705344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158c374-b559-4242-8637-5504e9effa4d None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:20.717184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-d158c374-b559-4242-8637-5504e9effa4d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:28:20.717184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:20.717184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:20.717184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:20.717184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:20.717184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:20.717184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:20.717184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:20.717184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:20.717184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:20.717184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:20.717184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:20.717184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:28:20.717184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:28:20.717184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:28:20.717184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:28:20.718423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:20.718423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:20.718423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:28:20.718423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:28:20.718423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:28:20.718423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:28:20.718423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:20.718423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:20.718423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:20.718423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:20.718423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:20.718423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:20.718423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:20.718423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:20.718423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:20.718423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:20.718423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:20.718423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:20.719666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:20.719666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:20.719666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:28:20.719666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:28:20.719666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:28:20.719666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:28:20.719666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:28:20.719666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:28:20.719666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:28:20.719666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:28:20.719666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 650/1267] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 22:28:20.724552 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275c7ac-ed78-471f-91f3-33eed59d981a None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:20.827035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275c7ac-ed78-471f-91f3-33eed59d981a None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:20.861368 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b723f-69aa-4fbb-ab95-570240df4877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:20.862000 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b723f-69aa-4fbb-ab95-570240df4877 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:20.862272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b723f-69aa-4fbb-ab95-570240df4877 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:20.862538 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b723f-69aa-4fbb-ab95-570240df4877 None admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:20.863802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b723f-69aa-4fbb-ab95-570240df4877 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:20.864093 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b723f-69aa-4fbb-ab95-570240df4877 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:20.865556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b723f-69aa-4fbb-ab95-570240df4877 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:20.874768 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:20.884085 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 618/1268] 10.222.0.59 () {56 vars in 1143 bytes} [Mon Aug 29 22:28:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:20.891268 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158c374-b559-4242-8637-5504e9effa4d None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:20.997455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158c374-b559-4242-8637-5504e9effa4d None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:21.031368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5b3bc-0087-4b7b-92bf-4b29d3db3f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:21.031983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5b3bc-0087-4b7b-92bf-4b29d3db3f1c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:21.032286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5b3bc-0087-4b7b-92bf-4b29d3db3f1c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:21.032553 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5b3bc-0087-4b7b-92bf-4b29d3db3f1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:21.043270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5b3bc-0087-4b7b-92bf-4b29d3db3f1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:21.044902 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5b3bc-0087-4b7b-92bf-4b29d3db3f1c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:21.059242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-43a5b3bc-0087-4b7b-92bf-4b29d3db3f1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:21.059242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:21.059242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:21.059242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:21.059242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:21.059242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:21.059242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:21.059242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:21.059242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:21.059242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:21.059242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:21.059242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:21.059242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:28:21.059242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:28:21.059242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:28:21.059242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:28:21.060574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:21.060574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:21.060574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:21.060574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:21.060574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:21.060574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:21.060574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:21.060574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:21.060574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:21.060574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:21.060574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:21.060574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:21.060574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:21.060574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:21.060574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:21.060574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:21.060574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:28:21.061697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:28:21.061697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:21.061697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:28:21.061697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 651/1269] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:21.067267 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b723f-69aa-4fbb-ab95-570240df4877 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:21.170523 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b723f-69aa-4fbb-ab95-570240df4877 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:21.206069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa940ac7-950e-4d6c-bf38-7a05a560e9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:21.206726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa940ac7-950e-4d6c-bf38-7a05a560e9c8 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:21.207148 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa940ac7-950e-4d6c-bf38-7a05a560e9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:21.207418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa940ac7-950e-4d6c-bf38-7a05a560e9c8 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:21.208062 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa940ac7-950e-4d6c-bf38-7a05a560e9c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:21.208388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa940ac7-950e-4d6c-bf38-7a05a560e9c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:21.209748 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa940ac7-950e-4d6c-bf38-7a05a560e9c8 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:21.221473 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 619/1270] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:28:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:21.229501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5b3bc-0087-4b7b-92bf-4b29d3db3f1c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:21.261607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:21.333198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5b3bc-0087-4b7b-92bf-4b29d3db3f1c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:21.371191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08fe237-8c8f-4568-8d52-b34305793a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:21.371814 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fe237-8c8f-4568-8d52-b34305793a62 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:21.372091 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fe237-8c8f-4568-8d52-b34305793a62 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:21.372391 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fe237-8c8f-4568-8d52-b34305793a62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:21.379175 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08fe237-8c8f-4568-8d52-b34305793a62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:21.380742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08fe237-8c8f-4568-8d52-b34305793a62 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:21.391995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-c08fe237-8c8f-4568-8d52-b34305793a62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:28:21.391995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:21.391995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:21.391995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:21.391995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:21.391995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:21.391995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:21.391995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:21.391995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:21.391995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:21.391995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:21.391995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:21.391995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:28:21.391995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:28:21.391995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:28:21.393277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:28:21.393277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:21.393277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:21.393277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:21.393277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:21.393277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:21.393277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:21.393277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:21.393277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:21.393277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:21.393277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:21.393277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:21.393277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:21.393277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:21.393277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:21.393277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:21.393277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:21.394517 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:28:21.394517 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:28:21.394517 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:28:21.394517 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:28:21.394517 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:28:21.394517 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:28:21.394517 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:28:21.394517 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:28:21.394517 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 652/1271] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:28:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:21.399478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa940ac7-950e-4d6c-bf38-7a05a560e9c8 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:21.462541 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:21.505325 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa940ac7-950e-4d6c-bf38-7a05a560e9c8 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:21.541717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b5ab1-2f0f-4ef4-91f8-67457f4cb105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:21.542395 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b5ab1-2f0f-4ef4-91f8-67457f4cb105 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:21.542590 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b5ab1-2f0f-4ef4-91f8-67457f4cb105 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:21.542858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b5ab1-2f0f-4ef4-91f8-67457f4cb105 None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:21.543552 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b5ab1-2f0f-4ef4-91f8-67457f4cb105 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:21.543848 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b5ab1-2f0f-4ef4-91f8-67457f4cb105 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:21.545301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b5ab1-2f0f-4ef4-91f8-67457f4cb105 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:21.557520 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 620/1272] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:28:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:21.563689 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08fe237-8c8f-4568-8d52-b34305793a62 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:21.658620 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:21.668515 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08fe237-8c8f-4568-8d52-b34305793a62 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:21.702055 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54ef80-837c-450f-98ef-45f4de765259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:21.702701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54ef80-837c-450f-98ef-45f4de765259 None admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:21.702978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54ef80-837c-450f-98ef-45f4de765259 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:21.703248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54ef80-837c-450f-98ef-45f4de765259 None admin] PATH_INFO: `/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/7215ee41033742d79671f4f42b81cd06/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:21.705165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54ef80-837c-450f-98ef-45f4de765259 None admin] RBAC: Authorizing `identity:create_grant(project_id=7799820fc7c64fd6a99a0a9ccf1338e3, user_id=7215ee41033742d79671f4f42b81cd06, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:21.706589 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54ef80-837c-450f-98ef-45f4de765259 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:21.803133 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 653/1273] 10.222.0.59 () {58 vars in 1439 bytes} [Mon Aug 29 22:28:21 2022] PUT /identity/v3/projects/7799820fc7c64fd6a99a0a9ccf1338e3/users/7215ee41033742d79671f4f42b81cd06/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:28:23.717597 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fcc8e-75f8-4700-98b8-314386e08688 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:23.718151 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fcc8e-75f8-4700-98b8-314386e08688 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:23.718254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fcc8e-75f8-4700-98b8-314386e08688 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:23.719579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 621/1274] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:23.726188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902b606-eda4-46cd-ad6d-2121b2d42a21 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:23.726188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902b606-eda4-46cd-ad6d-2121b2d42a21 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:23.726188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902b606-eda4-46cd-ad6d-2121b2d42a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:23.744429 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-c902b606-eda4-46cd-ad6d-2121b2d42a21 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:23.745612 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c902b606-eda4-46cd-ad6d-2121b2d42a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:23.813508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 654/1275] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:28:24.082833 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fcc8e-75f8-4700-98b8-314386e08688 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:24.188235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fcc8e-75f8-4700-98b8-314386e08688 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:24.208869 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:24.226943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb531f-7cb8-41c9-aa05-3e6ba52c6a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:24.227576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb531f-7cb8-41c9-aa05-3e6ba52c6a52 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:24.227853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb531f-7cb8-41c9-aa05-3e6ba52c6a52 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:24.228120 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb531f-7cb8-41c9-aa05-3e6ba52c6a52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:24.239629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb531f-7cb8-41c9-aa05-3e6ba52c6a52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 22:28:24.244494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-badb531f-7cb8-41c9-aa05-3e6ba52c6a52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:24.244494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:24.244494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:24.244494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:24.244494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:24.244494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:24.244494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:24.244494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:24.244494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:24.244494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:24.244494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:24.244494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:24.244494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 22:28:24.244494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 22:28:24.244494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 22:28:24.244494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 22:28:24.245622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 22:28:24.245622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 22:28:24.245622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 22:28:24.245622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 22:28:24.245622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:24.245622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:24.245622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:24.245622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:24.245622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:24.245622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:24.245622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:24.245622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:24.245622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:24.245622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:24.245622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:24.245622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:24.246920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:24.246920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:24.246920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:24.246920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:24.246920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 22:28:24.246920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 22:28:24.246920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 22:28:24.246920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 22:28:24.246920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 22:28:24.246920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 22:28:24.246920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 22:28:24.246920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:24.246920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 622/1276] 10.222.0.59 () {56 vars in 1096 bytes} [Mon Aug 29 22:28:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:24.251943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902b606-eda4-46cd-ad6d-2121b2d42a21 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:24.335714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:24.355502 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902b606-eda4-46cd-ad6d-2121b2d42a21 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:24.391443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c330fe-9881-4281-836d-ba73ee57cc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:24.392170 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c330fe-9881-4281-836d-ba73ee57cc6e None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:24.392458 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c330fe-9881-4281-836d-ba73ee57cc6e None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:24.392751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c330fe-9881-4281-836d-ba73ee57cc6e None admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:24.393411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c330fe-9881-4281-836d-ba73ee57cc6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:24.393710 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c330fe-9881-4281-836d-ba73ee57cc6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:24.395509 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c330fe-9881-4281-836d-ba73ee57cc6e None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:24.404684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 655/1277] 10.222.0.59 () {56 vars in 1099 bytes} [Mon Aug 29 22:28:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:24.411420 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb531f-7cb8-41c9-aa05-3e6ba52c6a52 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:24.515809 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb531f-7cb8-41c9-aa05-3e6ba52c6a52 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:24.555311 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abfc16-a038-47e9-99f7-df82520c553a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:24.555947 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abfc16-a038-47e9-99f7-df82520c553a None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:24.556282 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abfc16-a038-47e9-99f7-df82520c553a None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:24.556515 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abfc16-a038-47e9-99f7-df82520c553a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:24.568259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28abfc16-a038-47e9-99f7-df82520c553a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:24.569767 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28abfc16-a038-47e9-99f7-df82520c553a None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:24.585352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-28abfc16-a038-47e9-99f7-df82520c553a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:24.585352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:24.585352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:24.585352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:24.585352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:24.585352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:24.585352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:24.585352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:24.585352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:24.585352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:24.585352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:24.585352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:24.585352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:28:24.585352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:28:24.585352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:28:24.585352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:28:24.586629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:24.586629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:24.586629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:24.586629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:24.586629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:24.586629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:24.586629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:24.586629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:24.586629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:24.586629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:24.586629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:24.586629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:24.586629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:24.586629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:24.586629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:24.586629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:24.586629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:28:24.587816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:28:24.587816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:24.587816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:24.587816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 623/1278] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:24.592572 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c330fe-9881-4281-836d-ba73ee57cc6e None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:24.702084 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c330fe-9881-4281-836d-ba73ee57cc6e None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:24.736906 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e91f02-d140-483d-b076-b81bb27321a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:24.737689 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e91f02-d140-483d-b076-b81bb27321a4 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:24.737864 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e91f02-d140-483d-b076-b81bb27321a4 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:24.738151 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e91f02-d140-483d-b076-b81bb27321a4 None admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:24.738792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e91f02-d140-483d-b076-b81bb27321a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:24.739090 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e91f02-d140-483d-b076-b81bb27321a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:24.740487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e91f02-d140-483d-b076-b81bb27321a4 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:24.752491 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 656/1279] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:28:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:24.758369 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abfc16-a038-47e9-99f7-df82520c553a None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:24.790554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:24.863766 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abfc16-a038-47e9-99f7-df82520c553a None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:24.901825 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79459d-3be2-4079-9dd3-20e632645372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:24.902110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79459d-3be2-4079-9dd3-20e632645372 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:24.902277 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79459d-3be2-4079-9dd3-20e632645372 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:24.902539 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79459d-3be2-4079-9dd3-20e632645372 None admin] PATH_INFO: `/v3/users/placement` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:24.910922 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79459d-3be2-4079-9dd3-20e632645372 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:24.912368 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79459d-3be2-4079-9dd3-20e632645372 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:24.923799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-9d79459d-3be2-4079-9dd3-20e632645372 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:28:24.923799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:24.923799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:24.923799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:24.923799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:24.923799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:24.923799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:24.923799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:24.923799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:24.923799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:24.923799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:24.923799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:24.923799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 22:28:24.923799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 22:28:24.923799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 22:28:24.923799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 22:28:24.927327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:24.927327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:24.927327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 22:28:24.927327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:28:24.927327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 22:28:24.927327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 22:28:24.927327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:24.927327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:24.927327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:24.927327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:24.927327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:24.927327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:24.927327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:24.927327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:24.927327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:24.927327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:24.927327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:24.927327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:24.929070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:24.929070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:24.929070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 22:28:24.929070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 22:28:24.929070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 22:28:24.929070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 22:28:24.929070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 22:28:24.929070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 22:28:24.929070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 22:28:24.929070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:24.929070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 624/1280] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:24.931648 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e91f02-d140-483d-b076-b81bb27321a4 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:24.935699 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:25.033325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e91f02-d140-483d-b076-b81bb27321a4 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:25.068079 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931eb4ae-e9fa-427b-b731-d26812944403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:25.068753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931eb4ae-e9fa-427b-b731-d26812944403 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:25.069024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931eb4ae-e9fa-427b-b731-d26812944403 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:25.069290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931eb4ae-e9fa-427b-b731-d26812944403 None admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:25.070004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931eb4ae-e9fa-427b-b731-d26812944403 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:25.070302 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931eb4ae-e9fa-427b-b731-d26812944403 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:25.071711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931eb4ae-e9fa-427b-b731-d26812944403 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:25.089482 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 657/1281] 10.222.0.59 () {56 vars in 1143 bytes} [Mon Aug 29 22:28:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:25.095496 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79459d-3be2-4079-9dd3-20e632645372 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:25.194255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:25.203895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79459d-3be2-4079-9dd3-20e632645372 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:25.242166 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38654d-cdfa-49cb-a5b0-25bf6efe62df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:25.242776 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38654d-cdfa-49cb-a5b0-25bf6efe62df None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:25.243052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38654d-cdfa-49cb-a5b0-25bf6efe62df None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:25.243318 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38654d-cdfa-49cb-a5b0-25bf6efe62df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:25.322789 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38654d-cdfa-49cb-a5b0-25bf6efe62df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:25.324264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38654d-cdfa-49cb-a5b0-25bf6efe62df None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:25.416801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:25.428900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 625/1282] 10.222.0.59 () {58 vars in 1282 bytes} [Mon Aug 29 22:28:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 22:28:25.434900 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931eb4ae-e9fa-427b-b731-d26812944403 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:25.507849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:25.547394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931eb4ae-e9fa-427b-b731-d26812944403 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:25.583833 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca5245-d665-4a47-9f1f-f7db4bd3d9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:25.584491 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca5245-d665-4a47-9f1f-f7db4bd3d9f3 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:25.584773 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca5245-d665-4a47-9f1f-f7db4bd3d9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:25.585043 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca5245-d665-4a47-9f1f-f7db4bd3d9f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:25.595645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca5245-d665-4a47-9f1f-f7db4bd3d9f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:25.597771 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca5245-d665-4a47-9f1f-f7db4bd3d9f3 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:25.616208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-0fca5245-d665-4a47-9f1f-f7db4bd3d9f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:25.616208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:25.616208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:25.616208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:25.616208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:25.616208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:25.616208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:25.616208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:25.616208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:25.616208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:25.616208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:25.616208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:25.616208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 22:28:25.616208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 22:28:25.616208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 22:28:25.616208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 22:28:25.622481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:25.622481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:25.622481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:25.622481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:25.622481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:25.622481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:25.622481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:25.622481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:25.622481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:25.622481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:25.622481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:25.622481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:25.622481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:25.622481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:25.622481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:25.622481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:25.622481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 22:28:25.623662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 22:28:25.623662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 22:28:25.623662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:28:25.623662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 658/1283] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:28:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:25.623895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38654d-cdfa-49cb-a5b0-25bf6efe62df None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:25.724369 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38654d-cdfa-49cb-a5b0-25bf6efe62df None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:25.761379 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67ce6c-12fc-463a-a17f-b9e17d03c714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:25.762036 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67ce6c-12fc-463a-a17f-b9e17d03c714 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:25.762163 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67ce6c-12fc-463a-a17f-b9e17d03c714 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:25.762430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67ce6c-12fc-463a-a17f-b9e17d03c714 None admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:25.763079 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67ce6c-12fc-463a-a17f-b9e17d03c714 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:25.763378 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67ce6c-12fc-463a-a17f-b9e17d03c714 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:25.764778 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67ce6c-12fc-463a-a17f-b9e17d03c714 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:25.777830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 626/1284] 10.222.0.59 () {56 vars in 1109 bytes} [Mon Aug 29 22:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:25.783721 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca5245-d665-4a47-9f1f-f7db4bd3d9f3 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:25.891860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca5245-d665-4a47-9f1f-f7db4bd3d9f3 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:25.907520 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:25.932967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc7b26-98c0-41f5-ae6e-b365c59ed027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:25.933950 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc7b26-98c0-41f5-ae6e-b365c59ed027 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:25.934342 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc7b26-98c0-41f5-ae6e-b365c59ed027 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:25.934711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc7b26-98c0-41f5-ae6e-b365c59ed027 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:25.943859 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc7b26-98c0-41f5-ae6e-b365c59ed027 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:25.945380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc7b26-98c0-41f5-ae6e-b365c59ed027 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:25.958688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-68bc7b26-98c0-41f5-ae6e-b365c59ed027 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:28:25.958688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:25.958688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:25.958688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:25.958688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:25.958688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:25.958688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:25.958688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:25.958688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:25.958688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:25.958688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:25.958688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:25.958688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 22:28:25.958688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 22:28:25.958688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 22:28:25.960098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 22:28:25.960098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:25.960098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:25.960098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:25.960098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:25.960098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:25.960098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:25.960098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:25.960098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:25.960098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:25.960098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:25.960098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:25.960098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:25.960098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:25.960098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:25.960098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:25.960098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:25.961256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 22:28:25.961256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 22:28:25.961256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 22:28:25.961256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 22:28:25.961256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 22:28:25.961256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 22:28:25.961256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 22:28:25.961256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 22:28:25.961256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 659/1285] 10.222.0.59 () {56 vars in 1111 bytes} [Mon Aug 29 22:28:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:25.966874 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67ce6c-12fc-463a-a17f-b9e17d03c714 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:26.037057 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:26.096547 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67ce6c-12fc-463a-a17f-b9e17d03c714 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:26.139246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80979104-b0d0-4796-9b20-d51f1a56cb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:26.139880 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80979104-b0d0-4796-9b20-d51f1a56cb81 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:26.140182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80979104-b0d0-4796-9b20-d51f1a56cb81 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:26.140450 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80979104-b0d0-4796-9b20-d51f1a56cb81 None admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:26.141171 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80979104-b0d0-4796-9b20-d51f1a56cb81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:26.141461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80979104-b0d0-4796-9b20-d51f1a56cb81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:26.142915 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80979104-b0d0-4796-9b20-d51f1a56cb81 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:26.164178 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 627/1286] 10.222.0.59 () {56 vars in 1148 bytes} [Mon Aug 29 22:28:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:26.165566 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc7b26-98c0-41f5-ae6e-b365c59ed027 None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:26.293677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc7b26-98c0-41f5-ae6e-b365c59ed027 None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:26.332650 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9b5ba-7cbf-45ce-ac72-3a7a682f3545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:26.333277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9b5ba-7cbf-45ce-ac72-3a7a682f3545 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:26.333552 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9b5ba-7cbf-45ce-ac72-3a7a682f3545 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:26.333823 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9b5ba-7cbf-45ce-ac72-3a7a682f3545 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:26.334554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9b5ba-7cbf-45ce-ac72-3a7a682f3545 None admin] RBAC: Adding query filter params (role.id=a836108447dc439e994611bc74d53d9c, scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3, user.id=7215ee41033742d79671f4f42b81cd06) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:26.334860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9b5ba-7cbf-45ce-ac72-3a7a682f3545 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:26.336323 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9b5ba-7cbf-45ce-ac72-3a7a682f3545 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:26.343438 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 660/1287] 10.222.0.59 () {56 vars in 1374 bytes} [Mon Aug 29 22:28:26 2022] GET /identity/v3/role_assignments?user.id=7215ee41033742d79671f4f42b81cd06&scope.project.id=7799820fc7c64fd6a99a0a9ccf1338e3&role.id=a836108447dc439e994611bc74d53d9c => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:28.371175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85036cea-622f-4ae8-b307-5954370cc7ce None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:28.371741 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85036cea-622f-4ae8-b307-5954370cc7ce None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:28.371832 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85036cea-622f-4ae8-b307-5954370cc7ce None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:28.373199 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 628/1288] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:28:28.380238 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846faa5-2a63-4308-9b3b-197c6825d8a9 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:28.380727 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846faa5-2a63-4308-9b3b-197c6825d8a9 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:28.380816 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846faa5-2a63-4308-9b3b-197c6825d8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:28.399199 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-e846faa5-2a63-4308-9b3b-197c6825d8a9 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:28.400413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e846faa5-2a63-4308-9b3b-197c6825d8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:28.430330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:28.469403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 661/1289] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:28:28.747302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85036cea-622f-4ae8-b307-5954370cc7ce None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:28.857704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85036cea-622f-4ae8-b307-5954370cc7ce None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:28.870679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:28.896350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be31a4-3e1a-48db-9578-9ee84f7f9733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:28.896997 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be31a4-3e1a-48db-9578-9ee84f7f9733 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:28.897293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be31a4-3e1a-48db-9578-9ee84f7f9733 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:28.897567 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be31a4-3e1a-48db-9578-9ee84f7f9733 None admin] PATH_INFO: `/v3/services/placement` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:28.898154 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be31a4-3e1a-48db-9578-9ee84f7f9733 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:28.899536 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be31a4-3e1a-48db-9578-9ee84f7f9733 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:28.912554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-08be31a4-3e1a-48db-9578-9ee84f7f9733 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 22:28:28.912554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:28.912554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:28:28.912554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:28:28.912554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:28:28.912554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:28:28.912554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:28:28.912554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:28:28.912554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 22:28:28.912554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:28.912554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:28:28.912554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:28.912554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:28.912554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:28.912554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:28.912554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:28.912554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:28.917405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:28.917405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:28.917405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:28.917405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:28.917405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:28.917405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:28.917405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:28:28.917405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:28:28.917405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:28:28.917405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:28:28.917405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:28.917405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:28.917405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:28.917405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:28.917405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:28.917405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:28.918272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:28.918272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:28.918272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:28.918272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:28.918272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:28.918272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:28.918272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:28.918272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:28.918272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:28.918272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:28.918272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:28:28.918272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:28:28.918272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 22:28:28.918272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:28.918272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 629/1290] 10.222.0.59 () {56 vars in 1117 bytes} [Mon Aug 29 22:28:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:28.920791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846faa5-2a63-4308-9b3b-197c6825d8a9 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:29.032045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846faa5-2a63-4308-9b3b-197c6825d8a9 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:29.073503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227fead-ac58-4455-8986-9c34d663062a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:29.074108 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227fead-ac58-4455-8986-9c34d663062a None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:29.074376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227fead-ac58-4455-8986-9c34d663062a None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:29.074645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227fead-ac58-4455-8986-9c34d663062a None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:29.075271 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227fead-ac58-4455-8986-9c34d663062a None admin] RBAC: Adding query filter params (name=placement) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:29.075566 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227fead-ac58-4455-8986-9c34d663062a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:29.077237 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227fead-ac58-4455-8986-9c34d663062a None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:29.083004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 662/1291] 10.222.0.59 () {56 vars in 1116 bytes} [Mon Aug 29 22:28:28 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:29.089215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be31a4-3e1a-48db-9578-9ee84f7f9733 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:29.223226 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be31a4-3e1a-48db-9578-9ee84f7f9733 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:29.263646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2c619-0a32-43f3-abbf-f22f285fee3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:29.264294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2c619-0a32-43f3-abbf-f22f285fee3c None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:29.264571 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2c619-0a32-43f3-abbf-f22f285fee3c None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:29.264843 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2c619-0a32-43f3-abbf-f22f285fee3c None admin] PATH_INFO: `/v3/services` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:29.265480 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2c619-0a32-43f3-abbf-f22f285fee3c None admin] RBAC: Adding query filter params (type=placement) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:29.265813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2c619-0a32-43f3-abbf-f22f285fee3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:29.267147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2c619-0a32-43f3-abbf-f22f285fee3c None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:29.272607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 630/1292] 10.222.0.59 () {56 vars in 1116 bytes} [Mon Aug 29 22:28:29 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:31.242518 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7816bd-9cd0-439f-9deb-b9a04484aba5 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:31.243052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7816bd-9cd0-439f-9deb-b9a04484aba5 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:31.243161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7816bd-9cd0-439f-9deb-b9a04484aba5 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:31.244740 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 663/1293] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:28:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:31.250327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05c6c4-298a-4cf4-a5c1-7ff6e5a74b7f None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:31.250729 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05c6c4-298a-4cf4-a5c1-7ff6e5a74b7f None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:31.250862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05c6c4-298a-4cf4-a5c1-7ff6e5a74b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:31.270867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-ef05c6c4-298a-4cf4-a5c1-7ff6e5a74b7f None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:31.272031 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ef05c6c4-298a-4cf4-a5c1-7ff6e5a74b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:31.301980 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:31.339665 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 631/1294] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:28:31.628893 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7816bd-9cd0-439f-9deb-b9a04484aba5 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:31.661031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:31.735041 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7816bd-9cd0-439f-9deb-b9a04484aba5 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:31.770092 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feaec35-aa92-43cf-82e5-fb1ccebd2a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:31.770682 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feaec35-aa92-43cf-82e5-fb1ccebd2a5a None admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:31.770951 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feaec35-aa92-43cf-82e5-fb1ccebd2a5a None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:31.771213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feaec35-aa92-43cf-82e5-fb1ccebd2a5a None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:31.771851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feaec35-aa92-43cf-82e5-fb1ccebd2a5a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:31.773226 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feaec35-aa92-43cf-82e5-fb1ccebd2a5a None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:31.848199 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 664/1295] 10.222.0.59 () {60 vars in 1141 bytes} [Mon Aug 29 22:28:31 2022] POST /identity/v3/services => generated 246 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:28:33.790126 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed35e9-2bd7-49d3-83c6-d75417e3836e None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:33.790689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed35e9-2bd7-49d3-83c6-d75417e3836e None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:33.790689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed35e9-2bd7-49d3-83c6-d75417e3836e None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:33.791983 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 632/1296] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:33.797332 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f83b8-41c5-4ca4-8bcc-ba1a8aab5442 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:33.797758 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f83b8-41c5-4ca4-8bcc-ba1a8aab5442 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:33.797863 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f83b8-41c5-4ca4-8bcc-ba1a8aab5442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:33.817088 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-f97f83b8-41c5-4ca4-8bcc-ba1a8aab5442 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:33.819857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f97f83b8-41c5-4ca4-8bcc-ba1a8aab5442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:33.894525 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 665/1297] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:33 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:28:34.171116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed35e9-2bd7-49d3-83c6-d75417e3836e None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:34.275139 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed35e9-2bd7-49d3-83c6-d75417e3836e None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:34.311564 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299797f-ad76-4c79-a9e3-2481bbd37019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:34.312221 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299797f-ad76-4c79-a9e3-2481bbd37019 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:34.312497 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299797f-ad76-4c79-a9e3-2481bbd37019 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:34.312763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299797f-ad76-4c79-a9e3-2481bbd37019 None admin] PATH_INFO: `/v3/services/placement` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:34.313339 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299797f-ad76-4c79-a9e3-2481bbd37019 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:34.314676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299797f-ad76-4c79-a9e3-2481bbd37019 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:34.323635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-f299797f-ad76-4c79-a9e3-2481bbd37019 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 22:28:34.323635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:34.323635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:28:34.323635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:28:34.323635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:28:34.323635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:28:34.323635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:28:34.323635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:28:34.323635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 22:28:34.323635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:34.323635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:28:34.323635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:34.323635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:34.323635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:34.323635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:34.323635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:34.323635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:34.328428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:34.328428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:34.328428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:34.328428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:34.328428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:34.328428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:34.328428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:28:34.328428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:28:34.328428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:28:34.328428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:28:34.328428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:34.328428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:34.328428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:34.328428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:34.328428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:34.328428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:34.329636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:34.329636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:34.329636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:34.329636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:34.329636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:34.329636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:34.329636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:34.329636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:34.329636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:34.329636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:34.329636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:28:34.329636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:28:34.329636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 22:28:34.329636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:34.329636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 633/1298] 10.222.0.59 () {56 vars in 1117 bytes} [Mon Aug 29 22:28:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:34.331232 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f83b8-41c5-4ca4-8bcc-ba1a8aab5442 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:34.350165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:34.432718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f83b8-41c5-4ca4-8bcc-ba1a8aab5442 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:34.467794 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a1510-66ff-4029-9a16-07ac5892181c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:34.468495 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a1510-66ff-4029-9a16-07ac5892181c None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:34.468772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a1510-66ff-4029-9a16-07ac5892181c None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:34.469042 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a1510-66ff-4029-9a16-07ac5892181c None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:34.469665 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a1510-66ff-4029-9a16-07ac5892181c None admin] RBAC: Adding query filter params (name=placement) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:34.469962 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a1510-66ff-4029-9a16-07ac5892181c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:34.471321 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a1510-66ff-4029-9a16-07ac5892181c None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:34.476796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 666/1299] 10.222.0.59 () {56 vars in 1116 bytes} [Mon Aug 29 22:28:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:34.483095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299797f-ad76-4c79-a9e3-2481bbd37019 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:34.514941 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:34.583097 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299797f-ad76-4c79-a9e3-2481bbd37019 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:34.620075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aba940-3466-4ec1-aa6b-7df4b9cd2891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:34.620709 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aba940-3466-4ec1-aa6b-7df4b9cd2891 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:34.620985 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aba940-3466-4ec1-aa6b-7df4b9cd2891 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:34.621250 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aba940-3466-4ec1-aa6b-7df4b9cd2891 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:34.621953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aba940-3466-4ec1-aa6b-7df4b9cd2891 None admin] RBAC: Adding query filter params (interface=public, service_id=f32482b774cd49f4a6d8822b589748b2, region_id=RegionOne) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:34.622279 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aba940-3466-4ec1-aa6b-7df4b9cd2891 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:34.623626 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aba940-3466-4ec1-aa6b-7df4b9cd2891 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:34.629656 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 634/1300] 10.222.0.59 () {56 vars in 1251 bytes} [Mon Aug 29 22:28:34 2022] GET /identity/v3/endpoints?service_id=f32482b774cd49f4a6d8822b589748b2&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:28:34.635218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a1510-66ff-4029-9a16-07ac5892181c None admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:34.720883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:34.740873 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a1510-66ff-4029-9a16-07ac5892181c None admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:34.776869 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67741783-4aa2-4ce2-bf44-4e0c07c08840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:34.777466 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67741783-4aa2-4ce2-bf44-4e0c07c08840 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:34.777743 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67741783-4aa2-4ce2-bf44-4e0c07c08840 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:34.778016 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67741783-4aa2-4ce2-bf44-4e0c07c08840 None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:34.778616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67741783-4aa2-4ce2-bf44-4e0c07c08840 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:34.780035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67741783-4aa2-4ce2-bf44-4e0c07c08840 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:34.787101 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 667/1301] 10.222.0.59 () {56 vars in 1087 bytes} [Mon Aug 29 22:28:34 2022] GET /identity/v3/services => generated 2191 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:28:36.721064 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2a26e-0c4e-4d82-98fb-5e9fe426e39b None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:36.722237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2a26e-0c4e-4d82-98fb-5e9fe426e39b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:36.722237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2a26e-0c4e-4d82-98fb-5e9fe426e39b None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:36.723557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 635/1302] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:28:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:28:36.729386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8c705-856d-4935-bef4-30744a711f63 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:36.729893 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8c705-856d-4935-bef4-30744a711f63 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:36.729893 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8c705-856d-4935-bef4-30744a711f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:36.749552 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-69f8c705-856d-4935-bef4-30744a711f63 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:28:36.750732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-69f8c705-856d-4935-bef4-30744a711f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:28:36.819560 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 668/1303] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:28:36 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 22:28:37.089350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2a26e-0c4e-4d82-98fb-5e9fe426e39b None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:37.187328 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:37.194331 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2a26e-0c4e-4d82-98fb-5e9fe426e39b None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:37.233144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1673f69b-7d68-4c1e-b7c1-29bdf2bb4a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:37.233980 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673f69b-7d68-4c1e-b7c1-29bdf2bb4a20 None admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:37.234347 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673f69b-7d68-4c1e-b7c1-29bdf2bb4a20 None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:37.234347 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673f69b-7d68-4c1e-b7c1-29bdf2bb4a20 None admin] PATH_INFO: `/v3/services/placement` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:37.234911 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1673f69b-7d68-4c1e-b7c1-29bdf2bb4a20 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:37.236307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1673f69b-7d68-4c1e-b7c1-29bdf2bb4a20 None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:37.245438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-1673f69b-7d68-4c1e-b7c1-29bdf2bb4a20 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 22:28:37.245438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:37.245438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 22:28:37.245438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 22:28:37.245438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 22:28:37.245438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 22:28:37.245438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 22:28:37.245438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:28:37.245438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 22:28:37.245438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:37.245438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 22:28:37.245438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:37.245438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 22:28:37.245438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 22:28:37.245438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 22:28:37.245438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 22:28:37.245438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 22:28:37.246751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 22:28:37.246751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 22:28:37.246751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 22:28:37.246751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 22:28:37.246751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 22:28:37.246751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 22:28:37.246751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 22:28:37.246751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 22:28:37.246751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 22:28:37.246751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 22:28:37.246751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 22:28:37.246751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 22:28:37.246751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 22:28:37.246751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 22:28:37.246751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 22:28:37.246751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 22:28:37.247671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 22:28:37.247671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 22:28:37.247671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 22:28:37.247671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 22:28:37.247671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 22:28:37.247671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 22:28:37.247671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 22:28:37.247671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 22:28:37.247671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 22:28:37.247671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 22:28:37.247671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 22:28:37.247671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 22:28:37.247671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 22:28:37.247671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 22:28:37.247671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 636/1304] 10.222.0.59 () {56 vars in 1117 bytes} [Mon Aug 29 22:28:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 22:28:37.253236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8c705-856d-4935-bef4-30744a711f63 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:37.335397 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:37.358965 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8c705-856d-4935-bef4-30744a711f63 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:37.392527 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7156920-da6f-471f-a9c1-caa5dea6b7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:37.393123 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7156920-da6f-471f-a9c1-caa5dea6b7c3 None admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:37.394401 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7156920-da6f-471f-a9c1-caa5dea6b7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:37.394664 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7156920-da6f-471f-a9c1-caa5dea6b7c3 None admin] PATH_INFO: `/v3/services` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:37.395283 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7156920-da6f-471f-a9c1-caa5dea6b7c3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:28:37.395578 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7156920-da6f-471f-a9c1-caa5dea6b7c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:37.396958 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7156920-da6f-471f-a9c1-caa5dea6b7c3 None admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:37.401957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 669/1305] 10.222.0.59 () {56 vars in 1116 bytes} [Mon Aug 29 22:28:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:28:37.407874 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1673f69b-7d68-4c1e-b7c1-29bdf2bb4a20 None admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:28:37.512086 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1673f69b-7d68-4c1e-b7c1-29bdf2bb4a20 None admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:28:37.547261 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5843c7c-64e7-41fd-a722-657e044420ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:28:37.547865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5843c7c-64e7-41fd-a722-657e044420ad None admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:28:37.548169 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5843c7c-64e7-41fd-a722-657e044420ad None admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:28:37.548499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5843c7c-64e7-41fd-a722-657e044420ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:28:37.549131 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5843c7c-64e7-41fd-a722-657e044420ad None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:28:37.551210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5843c7c-64e7-41fd-a722-657e044420ad None admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:28:37.631988 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 637/1306] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:28:37 2022] POST /identity/v3/endpoints => generated 331 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:28:47.436150 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=30 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:28:47.626291 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:13.925000 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ed39f-125c-4b45-ad3c-76f6cbbf201c None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:13.925000 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ed39f-125c-4b45-ad3c-76f6cbbf201c None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:13.925000 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ed39f-125c-4b45-ad3c-76f6cbbf201c None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:13.931422 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295c1c1-b9da-4b14-9097-5941f3dab33b None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:13.932384 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295c1c1-b9da-4b14-9097-5941f3dab33b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:13.932384 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295c1c1-b9da-4b14-9097-5941f3dab33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:13.935086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 670/1307] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:35:13 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:35:14.034054 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-b295c1c1-b9da-4b14-9097-5941f3dab33b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:35:14.035368 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b295c1c1-b9da-4b14-9097-5941f3dab33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:14.174991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 638/1308] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:35:14.512995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bba911-4c08-4913-8154-4e0261c15d08 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:14.513275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bba911-4c08-4913-8154-4e0261c15d08 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:14.513559 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bba911-4c08-4913-8154-4e0261c15d08 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:14.515314 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 671/1309] 10.222.0.59 () {54 vars in 965 bytes} [Mon Aug 29 22:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:35:14.562161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37836d52-bd85-4df9-8c9b-01bdad979bb0 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:14.562432 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37836d52-bd85-4df9-8c9b-01bdad979bb0 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:14.562781 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37836d52-bd85-4df9-8c9b-01bdad979bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:14.716695 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-37836d52-bd85-4df9-8c9b-01bdad979bb0 None None] MFA Rules not processed for user `cb802d01e71b486cb40c6388430d674d`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:35:14.718985 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-37836d52-bd85-4df9-8c9b-01bdad979bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:14.768837 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:14.884372 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 639/1310] 10.222.0.59 () {58 vars in 1062 bytes} [Mon Aug 29 22:35:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:35:14.888696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bba911-4c08-4913-8154-4e0261c15d08 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:35:14.890490 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-08bba911-4c08-4913-8154-4e0261c15d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:15.048664 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bba911-4c08-4913-8154-4e0261c15d08 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:35:15.152768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6cc93-1fd1-444b-9b17-4d6b6d2efbdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:35:15.153212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6cc93-1fd1-444b-9b17-4d6b6d2efbdc service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:15.153477 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6cc93-1fd1-444b-9b17-4d6b6d2efbdc service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:15.153738 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6cc93-1fd1-444b-9b17-4d6b6d2efbdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:15.155414 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-65b6cc93-1fd1-444b-9b17-4d6b6d2efbdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:15.231800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:15.326808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b6cc93-1fd1-444b-9b17-4d6b6d2efbdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:35:15.328454 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b6cc93-1fd1-444b-9b17-4d6b6d2efbdc service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:35:15.352591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 672/1311] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:35:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:35:17.945978 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2762d-0429-4b2a-a29a-41fd8fbfa32b None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:17.946554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2762d-0429-4b2a-a29a-41fd8fbfa32b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:17.946554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2762d-0429-4b2a-a29a-41fd8fbfa32b None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:17.947984 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 640/1312] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:35:17.953916 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41baf899-1973-44f7-9b3c-2ffdc8508b9b None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:17.954446 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41baf899-1973-44f7-9b3c-2ffdc8508b9b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:17.954446 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41baf899-1973-44f7-9b3c-2ffdc8508b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:18.034914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-41baf899-1973-44f7-9b3c-2ffdc8508b9b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:35:18.036685 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-41baf899-1973-44f7-9b3c-2ffdc8508b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:18.143342 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 673/1313] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:35:18.429387 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2762d-0429-4b2a-a29a-41fd8fbfa32b None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:35:18.431091 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-96c2762d-0429-4b2a-a29a-41fd8fbfa32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:18.555613 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2762d-0429-4b2a-a29a-41fd8fbfa32b None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:35:18.649149 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:18.652005 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b6cb6-a4dc-4d7e-b555-804dbfbe13a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:35:18.652660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b6cb6-a4dc-4d7e-b555-804dbfbe13a4 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:18.652934 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b6cb6-a4dc-4d7e-b555-804dbfbe13a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:18.653196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b6cb6-a4dc-4d7e-b555-804dbfbe13a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:18.654886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ce7b6cb6-a4dc-4d7e-b555-804dbfbe13a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:18.782406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b6cb6-a4dc-4d7e-b555-804dbfbe13a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:35:18.783913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b6cb6-a4dc-4d7e-b555-804dbfbe13a4 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:35:18.810250 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 641/1314] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:35:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:35:21.326990 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db420f17-d4ee-40ae-b97e-7c48f16160e6 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:21.327634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db420f17-d4ee-40ae-b97e-7c48f16160e6 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:21.327634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db420f17-d4ee-40ae-b97e-7c48f16160e6 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:21.328907 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 674/1315] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:35:21.334142 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bdf17-30dd-4579-82bc-9c8552130c84 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:21.334681 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bdf17-30dd-4579-82bc-9c8552130c84 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:21.334681 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bdf17-30dd-4579-82bc-9c8552130c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:21.403815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-4e0bdf17-30dd-4579-82bc-9c8552130c84 None None] MFA Rules not processed for user `98ba2c8a82264275b8635caae9c9737c`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:35:21.405084 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-4e0bdf17-30dd-4579-82bc-9c8552130c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:21.459168 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:21.505700 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 642/1316] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:35:21 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:35:21.796574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db420f17-d4ee-40ae-b97e-7c48f16160e6 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:35:21.798199 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-db420f17-d4ee-40ae-b97e-7c48f16160e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:21.808187 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:21.907916 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db420f17-d4ee-40ae-b97e-7c48f16160e6 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:35:21.984891 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106e810-fb95-4ac2-943a-def1c07750c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:35:21.985493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106e810-fb95-4ac2-943a-def1c07750c3 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:21.985762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106e810-fb95-4ac2-943a-def1c07750c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:21.986022 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106e810-fb95-4ac2-943a-def1c07750c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:21.987658 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-9106e810-fb95-4ac2-943a-def1c07750c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:22.050304 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:22.099425 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106e810-fb95-4ac2-943a-def1c07750c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:35:22.100896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106e810-fb95-4ac2-943a-def1c07750c3 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:35:22.121054 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 675/1317] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:35:21 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:35:25.700463 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71bbed-a526-417c-a2dc-9f5da40ceeb5 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:25.701059 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71bbed-a526-417c-a2dc-9f5da40ceeb5 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:25.701059 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71bbed-a526-417c-a2dc-9f5da40ceeb5 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:25.702336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 643/1318] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:35:25.709534 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b875d-f1e4-4ad1-be63-1b0e32a2c7e7 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:25.710069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b875d-f1e4-4ad1-be63-1b0e32a2c7e7 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:25.710069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b875d-f1e4-4ad1-be63-1b0e32a2c7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:25.782940 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-518b875d-f1e4-4ad1-be63-1b0e32a2c7e7 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:35:25.784191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-518b875d-f1e4-4ad1-be63-1b0e32a2c7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:25.900603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 676/1319] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:35:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:35:26.185569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71bbed-a526-417c-a2dc-9f5da40ceeb5 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:35:26.187148 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-bf71bbed-a526-417c-a2dc-9f5da40ceeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:26.298431 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71bbed-a526-417c-a2dc-9f5da40ceeb5 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:35:26.380938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94402b-7fb8-4a16-a5f6-29beead8248f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:35:26.381635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94402b-7fb8-4a16-a5f6-29beead8248f service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:26.381956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94402b-7fb8-4a16-a5f6-29beead8248f service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:26.382271 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94402b-7fb8-4a16-a5f6-29beead8248f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:26.384053 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-1d94402b-7fb8-4a16-a5f6-29beead8248f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:26.392920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:26.507648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94402b-7fb8-4a16-a5f6-29beead8248f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:35:26.509054 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94402b-7fb8-4a16-a5f6-29beead8248f service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:35:26.528911 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 644/1320] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:35:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:35:29.750157 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c7ad6-f05c-4654-975a-fa6aba428540 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:29.750755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c7ad6-f05c-4654-975a-fa6aba428540 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:29.750755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c7ad6-f05c-4654-975a-fa6aba428540 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:29.752236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 677/1321] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:35:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:35:29.758740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67527e-2855-458a-858e-40d825c4db61 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:29.759235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67527e-2855-458a-858e-40d825c4db61 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:29.759468 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67527e-2855-458a-858e-40d825c4db61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:29.839243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-ae67527e-2855-458a-858e-40d825c4db61 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:35:29.840278 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ae67527e-2855-458a-858e-40d825c4db61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:29.906919 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:29.954193 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 645/1322] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:35:30.265257 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46153938-fe06-496a-b912-7df35a8d13b9 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:30.265534 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46153938-fe06-496a-b912-7df35a8d13b9 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:30.265805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46153938-fe06-496a-b912-7df35a8d13b9 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:30.267240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 678/1323] 10.222.0.59 () {54 vars in 965 bytes} [Mon Aug 29 22:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:35:30.274397 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f550e7c-5f6b-48b8-a74a-982014d980ab None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:30.274693 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f550e7c-5f6b-48b8-a74a-982014d980ab None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:30.274966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f550e7c-5f6b-48b8-a74a-982014d980ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:30.361297 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-7f550e7c-5f6b-48b8-a74a-982014d980ab None None] MFA Rules not processed for user `cb802d01e71b486cb40c6388430d674d`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:35:30.362403 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-7f550e7c-5f6b-48b8-a74a-982014d980ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:30.449547 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 646/1324] 10.222.0.59 () {58 vars in 1062 bytes} [Mon Aug 29 22:35:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:35:30.459672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46153938-fe06-496a-b912-7df35a8d13b9 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:35:30.461400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-46153938-fe06-496a-b912-7df35a8d13b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:30.491196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:30.585980 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46153938-fe06-496a-b912-7df35a8d13b9 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:35:30.667987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1714f2-c641-4e8d-aa9f-f7b455486eb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:35:30.668621 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1714f2-c641-4e8d-aa9f-f7b455486eb8 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:30.668897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1714f2-c641-4e8d-aa9f-f7b455486eb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:30.669161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1714f2-c641-4e8d-aa9f-f7b455486eb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:30.670771 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-ad1714f2-c641-4e8d-aa9f-f7b455486eb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:30.753057 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:30.807295 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1714f2-c641-4e8d-aa9f-f7b455486eb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:35:30.808798 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1714f2-c641-4e8d-aa9f-f7b455486eb8 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:35:30.830750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 679/1325] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:35:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:35:33.146555 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb4e1c-bf59-4c55-8859-aff50ca85ea3 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:33.147147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb4e1c-bf59-4c55-8859-aff50ca85ea3 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:33.147147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb4e1c-bf59-4c55-8859-aff50ca85ea3 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:33.148527 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 647/1326] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:35:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:35:33.154223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131395d-d14a-4a07-b9e8-53373e7dc0bd None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:33.154772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131395d-d14a-4a07-b9e8-53373e7dc0bd None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:33.154906 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131395d-d14a-4a07-b9e8-53373e7dc0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:33.229535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-a131395d-d14a-4a07-b9e8-53373e7dc0bd None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:35:33.230759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a131395d-d14a-4a07-b9e8-53373e7dc0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:33.368027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 680/1327] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:35:33.735402 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb4e1c-bf59-4c55-8859-aff50ca85ea3 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:35:33.737186 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b8eb4e1c-bf59-4c55-8859-aff50ca85ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:33.822918 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:33.871515 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb4e1c-bf59-4c55-8859-aff50ca85ea3 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:35:33.963112 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cdabe-59e7-49b0-a749-a98fb54a3ff3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:35:33.963883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cdabe-59e7-49b0-a749-a98fb54a3ff3 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:33.964307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cdabe-59e7-49b0-a749-a98fb54a3ff3 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:33.964751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cdabe-59e7-49b0-a749-a98fb54a3ff3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:33.966552 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-2e0cdabe-59e7-49b0-a749-a98fb54a3ff3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:34.104048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:34.107015 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0cdabe-59e7-49b0-a749-a98fb54a3ff3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:35:34.108556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0cdabe-59e7-49b0-a749-a98fb54a3ff3 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:35:34.130324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 648/1328] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:35:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:35:36.213292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74407b5d-8cac-4483-9cf7-6393e391314e None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:36.213855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74407b5d-8cac-4483-9cf7-6393e391314e None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:36.214204 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74407b5d-8cac-4483-9cf7-6393e391314e None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:36.216124 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 681/1329] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:35:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:35:36.223243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e14a4b-c984-4150-8ec2-642ce5c95111 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:36.223829 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e14a4b-c984-4150-8ec2-642ce5c95111 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:36.223829 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e14a4b-c984-4150-8ec2-642ce5c95111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:36.301551 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-f9e14a4b-c984-4150-8ec2-642ce5c95111 None None] MFA Rules not processed for user `98ba2c8a82264275b8635caae9c9737c`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:35:36.302794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f9e14a4b-c984-4150-8ec2-642ce5c95111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:36.411793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 649/1330] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:35:36 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:35:36.700482 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74407b5d-8cac-4483-9cf7-6393e391314e None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:35:36.702061 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-74407b5d-8cac-4483-9cf7-6393e391314e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:36.746154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:36.846424 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74407b5d-8cac-4483-9cf7-6393e391314e None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:35:36.957824 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b1af8-69e6-4970-b612-dfe95ce6d9e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:35:36.958822 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b1af8-69e6-4970-b612-dfe95ce6d9e0 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:36.959118 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b1af8-69e6-4970-b612-dfe95ce6d9e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:36.959388 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b1af8-69e6-4970-b612-dfe95ce6d9e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:36.961198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8c3b1af8-69e6-4970-b612-dfe95ce6d9e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:37.074090 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:37.109417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b1af8-69e6-4970-b612-dfe95ce6d9e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:35:37.111143 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b1af8-69e6-4970-b612-dfe95ce6d9e0 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:35:37.130233 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 682/1331] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:35:36 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:35:39.569355 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090638c8-24e0-435a-ab9b-7c89fd5ff010 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:39.569887 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090638c8-24e0-435a-ab9b-7c89fd5ff010 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:39.569887 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090638c8-24e0-435a-ab9b-7c89fd5ff010 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:39.571437 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 650/1332] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:35:39.577260 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e6b43-bb27-4673-87aa-abd45e34ccdb None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:39.577260 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e6b43-bb27-4673-87aa-abd45e34ccdb None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:39.577836 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e6b43-bb27-4673-87aa-abd45e34ccdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:39.652113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-d66e6b43-bb27-4673-87aa-abd45e34ccdb None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:35:39.653375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-d66e6b43-bb27-4673-87aa-abd45e34ccdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:39.754620 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 683/1333] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:35:40.045437 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090638c8-24e0-435a-ab9b-7c89fd5ff010 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:35:40.051306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-090638c8-24e0-435a-ab9b-7c89fd5ff010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:40.121417 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:40.158450 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090638c8-24e0-435a-ab9b-7c89fd5ff010 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:35:40.248287 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc1029-9f19-478e-99f0-7d1d46d45ba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:35:40.248761 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc1029-9f19-478e-99f0-7d1d46d45ba1 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:40.249030 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc1029-9f19-478e-99f0-7d1d46d45ba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:40.249284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc1029-9f19-478e-99f0-7d1d46d45ba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:40.250973 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-59cc1029-9f19-478e-99f0-7d1d46d45ba1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:40.368561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc1029-9f19-478e-99f0-7d1d46d45ba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:35:40.369995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc1029-9f19-478e-99f0-7d1d46d45ba1 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:35:40.380152 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:40.391411 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 651/1334] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:35:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:35:42.455768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b0973-470b-456b-84c0-3de2b93ee541 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:42.456466 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b0973-470b-456b-84c0-3de2b93ee541 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:42.456466 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b0973-470b-456b-84c0-3de2b93ee541 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:42.457771 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 684/1335] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:35:42.463637 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65974a53-fae7-413e-82ef-320722eaf391 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:42.464228 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65974a53-fae7-413e-82ef-320722eaf391 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:42.464228 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65974a53-fae7-413e-82ef-320722eaf391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:42.538514 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-65974a53-fae7-413e-82ef-320722eaf391 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:35:42.539897 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-65974a53-fae7-413e-82ef-320722eaf391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:42.652359 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 652/1336] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:35:42.947174 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b0973-470b-456b-84c0-3de2b93ee541 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:35:42.948805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3a6b0973-470b-456b-84c0-3de2b93ee541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:42.996531 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:43.076217 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b0973-470b-456b-84c0-3de2b93ee541 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:35:43.169524 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3bf15e-6b90-46a5-b312-80979fa7b342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:35:43.170246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bf15e-6b90-46a5-b312-80979fa7b342 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:43.170660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bf15e-6b90-46a5-b312-80979fa7b342 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:43.171022 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bf15e-6b90-46a5-b312-80979fa7b342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:43.172992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3e3bf15e-6b90-46a5-b312-80979fa7b342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:43.272977 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:43.323459 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3bf15e-6b90-46a5-b312-80979fa7b342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:35:43.326229 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3bf15e-6b90-46a5-b312-80979fa7b342 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:35:43.360493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 685/1337] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:35:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:35:46.549230 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15146385-25cb-4e0d-8bba-b230c2c9d8f9 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:46.549759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15146385-25cb-4e0d-8bba-b230c2c9d8f9 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:46.549759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15146385-25cb-4e0d-8bba-b230c2c9d8f9 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:46.551738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 653/1338] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:35:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:35:46.557312 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352e784-f0af-4e3a-9bd8-a73372d34b6f None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:46.557765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352e784-f0af-4e3a-9bd8-a73372d34b6f None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:46.557877 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352e784-f0af-4e3a-9bd8-a73372d34b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:46.629919 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-3352e784-f0af-4e3a-9bd8-a73372d34b6f None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:35:46.631120 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3352e784-f0af-4e3a-9bd8-a73372d34b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:46.732369 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 686/1339] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:35:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:35:47.014142 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15146385-25cb-4e0d-8bba-b230c2c9d8f9 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:35:47.015772 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-15146385-25cb-4e0d-8bba-b230c2c9d8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:47.095600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:47.125567 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15146385-25cb-4e0d-8bba-b230c2c9d8f9 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:35:47.201177 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec4625-2d07-4331-a047-23b60568dd2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:35:47.201778 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec4625-2d07-4331-a047-23b60568dd2c service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:47.202047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec4625-2d07-4331-a047-23b60568dd2c service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:47.202310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec4625-2d07-4331-a047-23b60568dd2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:47.203890 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-c5ec4625-2d07-4331-a047-23b60568dd2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:47.319235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec4625-2d07-4331-a047-23b60568dd2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:35:47.320624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec4625-2d07-4331-a047-23b60568dd2c service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:35:47.339702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 654/1340] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:35:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:35:51.745056 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21faada0-3376-43d2-91c5-a58362558534 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:51.745621 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21faada0-3376-43d2-91c5-a58362558534 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:51.745621 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21faada0-3376-43d2-91c5-a58362558534 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:51.746990 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 687/1341] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:35:51.752858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2e85e-a288-4950-bbb8-361822fdd6a8 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:51.753976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2e85e-a288-4950-bbb8-361822fdd6a8 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:51.754147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2e85e-a288-4950-bbb8-361822fdd6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:51.759975 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:51.828681 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-d8f2e85e-a288-4950-bbb8-361822fdd6a8 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:35:51.829889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d8f2e85e-a288-4950-bbb8-361822fdd6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:51.942582 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 655/1342] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:35:52.227003 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21faada0-3376-43d2-91c5-a58362558534 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:35:52.228875 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-21faada0-3376-43d2-91c5-a58362558534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:52.278476 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:52.337987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21faada0-3376-43d2-91c5-a58362558534 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:35:52.422987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6f63a-33d9-4c6c-8711-f0cd2b71ca60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:35:52.423599 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6f63a-33d9-4c6c-8711-f0cd2b71ca60 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:52.423872 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6f63a-33d9-4c6c-8711-f0cd2b71ca60 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:52.424159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6f63a-33d9-4c6c-8711-f0cd2b71ca60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:52.426477 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-9dd6f63a-33d9-4c6c-8711-f0cd2b71ca60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:52.533485 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:52.551901 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6f63a-33d9-4c6c-8711-f0cd2b71ca60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:35:52.553396 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6f63a-33d9-4c6c-8711-f0cd2b71ca60 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:35:52.574052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 688/1343] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:35:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:35:55.609495 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b21349-fd7b-494b-99dd-a1fd53a07385 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:55.609998 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b21349-fd7b-494b-99dd-a1fd53a07385 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:55.609998 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b21349-fd7b-494b-99dd-a1fd53a07385 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:55.611490 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 656/1344] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:35:55.617694 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b326ff-b1e3-4a0a-81e1-fc33026efa96 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:55.618176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b326ff-b1e3-4a0a-81e1-fc33026efa96 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:55.618270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b326ff-b1e3-4a0a-81e1-fc33026efa96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:55.687381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-83b326ff-b1e3-4a0a-81e1-fc33026efa96 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:35:55.688777 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-83b326ff-b1e3-4a0a-81e1-fc33026efa96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:55.797572 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 689/1345] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:35:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:35:56.082445 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b21349-fd7b-494b-99dd-a1fd53a07385 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:35:56.083968 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-c6b21349-fd7b-494b-99dd-a1fd53a07385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:56.187918 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:35:56.207193 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b21349-fd7b-494b-99dd-a1fd53a07385 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:35:56.294465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff0325-d824-458c-91f0-c4fe7f21124c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:35:56.295075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff0325-d824-458c-91f0-c4fe7f21124c service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:35:56.295345 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff0325-d824-458c-91f0-c4fe7f21124c service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:35:56.295608 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff0325-d824-458c-91f0-c4fe7f21124c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:35:56.297219 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-93ff0325-d824-458c-91f0-c4fe7f21124c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:35:56.430180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff0325-d824-458c-91f0-c4fe7f21124c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:35:56.431641 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff0325-d824-458c-91f0-c4fe7f21124c service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:35:56.453480 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 657/1346] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:35:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:36:01.102385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d41c52-7e96-4d01-8dc7-4ff74561a083 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:01.103166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d41c52-7e96-4d01-8dc7-4ff74561a083 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:01.103315 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d41c52-7e96-4d01-8dc7-4ff74561a083 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:01.104795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 690/1347] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:01.110424 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10702d47-62d8-4add-8d7f-c7e1a1ce13a1 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:01.110929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10702d47-62d8-4add-8d7f-c7e1a1ce13a1 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:01.110929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10702d47-62d8-4add-8d7f-c7e1a1ce13a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:01.133164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:01.190462 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-10702d47-62d8-4add-8d7f-c7e1a1ce13a1 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:01.192061 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-10702d47-62d8-4add-8d7f-c7e1a1ce13a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:01.316989 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 658/1348] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:36:01.621518 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d41c52-7e96-4d01-8dc7-4ff74561a083 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:36:01.623120 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c2d41c52-7e96-4d01-8dc7-4ff74561a083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:01.689095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:01.739949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d41c52-7e96-4d01-8dc7-4ff74561a083 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:36:01.815902 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af8f73-1135-406f-b7a1-89d587a32bee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:36:01.816382 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af8f73-1135-406f-b7a1-89d587a32bee service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:01.816680 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af8f73-1135-406f-b7a1-89d587a32bee service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:01.816951 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af8f73-1135-406f-b7a1-89d587a32bee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:01.818552 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-94af8f73-1135-406f-b7a1-89d587a32bee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:01.925509 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:01.929161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af8f73-1135-406f-b7a1-89d587a32bee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:36:01.930603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af8f73-1135-406f-b7a1-89d587a32bee service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:36:01.950745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 691/1349] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:36:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:36:06.987465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa92952-4588-467a-bd06-8757f5a6cac7 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:06.988109 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa92952-4588-467a-bd06-8757f5a6cac7 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:06.988109 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa92952-4588-467a-bd06-8757f5a6cac7 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:06.989680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 659/1350] 10.222.0.59 () {54 vars in 921 bytes} [Mon Aug 29 22:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:06.997093 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f7ce1-eb67-424c-99ac-1643b4f32b55 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:06.997851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f7ce1-eb67-424c-99ac-1643b4f32b55 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:06.998276 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f7ce1-eb67-424c-99ac-1643b4f32b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:07.133305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-627f7ce1-eb67-424c-99ac-1643b4f32b55 None None] MFA Rules not processed for user `7215ee41033742d79671f4f42b81cd06`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:07.134715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-627f7ce1-eb67-424c-99ac-1643b4f32b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:07.258808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 692/1351] 10.222.0.59 () {58 vars in 1018 bytes} [Mon Aug 29 22:36:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:36:11.313737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=59 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:11.365360 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447ae02-1e6b-4a32-b840-98e9fa6c6313 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:11.365722 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447ae02-1e6b-4a32-b840-98e9fa6c6313 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:11.365927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447ae02-1e6b-4a32-b840-98e9fa6c6313 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:11.367395 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 660/1352] 10.222.0.59 () {54 vars in 921 bytes} [Mon Aug 29 22:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:11.372585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947a512-6deb-4fab-bd06-c8d466fcb11d None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:11.373536 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947a512-6deb-4fab-bd06-c8d466fcb11d None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:11.374016 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947a512-6deb-4fab-bd06-c8d466fcb11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:11.441936 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:11.470202 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-6947a512-6deb-4fab-bd06-c8d466fcb11d None None] MFA Rules not processed for user `7215ee41033742d79671f4f42b81cd06`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:11.472482 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6947a512-6deb-4fab-bd06-c8d466fcb11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:11.599893 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 693/1353] 10.222.0.59 () {58 vars in 1018 bytes} [Mon Aug 29 22:36:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:36:13.167671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98378c-f673-4698-9ee4-0d730eed6e82 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:13.168260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98378c-f673-4698-9ee4-0d730eed6e82 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:13.168260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98378c-f673-4698-9ee4-0d730eed6e82 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:13.169987 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 661/1354] 10.222.0.59 () {54 vars in 921 bytes} [Mon Aug 29 22:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:13.175397 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40d597-c57b-4b05-b28c-87d83409ae79 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:13.175888 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40d597-c57b-4b05-b28c-87d83409ae79 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:13.176064 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40d597-c57b-4b05-b28c-87d83409ae79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:13.254214 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-5c40d597-c57b-4b05-b28c-87d83409ae79 None None] MFA Rules not processed for user `7215ee41033742d79671f4f42b81cd06`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:13.255398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-5c40d597-c57b-4b05-b28c-87d83409ae79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:13.298286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:13.367895 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 694/1355] 10.222.0.59 () {58 vars in 1018 bytes} [Mon Aug 29 22:36:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:36:16.748980 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f34f1-c071-4952-87d3-087108c39c44 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:16.749528 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f34f1-c071-4952-87d3-087108c39c44 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:16.749528 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f34f1-c071-4952-87d3-087108c39c44 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:16.750848 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 662/1356] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:16.756310 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d584707-705f-46ab-b018-8bd37e5c380b None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:16.756739 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d584707-705f-46ab-b018-8bd37e5c380b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:16.756887 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d584707-705f-46ab-b018-8bd37e5c380b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:16.838070 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-6d584707-705f-46ab-b018-8bd37e5c380b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:16.840265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6d584707-705f-46ab-b018-8bd37e5c380b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:16.970553 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 695/1357] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:36:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:36:17.291962 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea6603-1ce2-479e-bb2a-4a928eb59e49 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:17.295370 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea6603-1ce2-479e-bb2a-4a928eb59e49 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:17.296049 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea6603-1ce2-479e-bb2a-4a928eb59e49 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:17.303457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 663/1358] 10.222.0.59 () {54 vars in 962 bytes} [Mon Aug 29 22:36:17 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:17.313183 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100ef29-c77e-4074-aacd-23a4d41a9de0 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:17.313585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100ef29-c77e-4074-aacd-23a4d41a9de0 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:17.313948 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100ef29-c77e-4074-aacd-23a4d41a9de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:17.335157 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:17.407057 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-5100ef29-c77e-4074-aacd-23a4d41a9de0 None None] MFA Rules not processed for user `7f21105679f245a19ea4850e6c010dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:17.408526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-5100ef29-c77e-4074-aacd-23a4d41a9de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:17.541421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 696/1359] 10.222.0.59 () {58 vars in 1059 bytes} [Mon Aug 29 22:36:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:36:17.551345 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea6603-1ce2-479e-bb2a-4a928eb59e49 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:36:17.553345 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-33ea6603-1ce2-479e-bb2a-4a928eb59e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:17.715528 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea6603-1ce2-479e-bb2a-4a928eb59e49 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:36:17.796630 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:17.840977 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff3596-31e3-4872-873a-8b1b6ed76548 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:36:17.841815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff3596-31e3-4872-873a-8b1b6ed76548 service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:17.842027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff3596-31e3-4872-873a-8b1b6ed76548 service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:17.842293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff3596-31e3-4872-873a-8b1b6ed76548 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:17.844088 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-19ff3596-31e3-4872-873a-8b1b6ed76548 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:17.947536 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff3596-31e3-4872-873a-8b1b6ed76548 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:36:17.949207 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff3596-31e3-4872-873a-8b1b6ed76548 service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:36:17.969985 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 664/1360] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:36:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:36:20.139957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329980f0-7dd9-4dd6-90e0-23a52274f0aa None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:20.140529 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329980f0-7dd9-4dd6-90e0-23a52274f0aa None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:20.140529 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329980f0-7dd9-4dd6-90e0-23a52274f0aa None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:20.142093 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 697/1361] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:20.148093 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75081805-077f-4a66-92f6-29e29c49f213 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:20.148587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75081805-077f-4a66-92f6-29e29c49f213 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:20.148666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75081805-077f-4a66-92f6-29e29c49f213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:20.220119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-75081805-077f-4a66-92f6-29e29c49f213 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:20.221275 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-75081805-077f-4a66-92f6-29e29c49f213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:20.262833 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:20.330272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 665/1362] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:36:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:36:20.632794 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329980f0-7dd9-4dd6-90e0-23a52274f0aa None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:36:20.633766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-329980f0-7dd9-4dd6-90e0-23a52274f0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:20.707818 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:20.802148 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329980f0-7dd9-4dd6-90e0-23a52274f0aa None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:36:20.937855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98753ee-91d7-4cff-bb0f-b6b0fca77438 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:36:20.938693 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98753ee-91d7-4cff-bb0f-b6b0fca77438 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:20.939072 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98753ee-91d7-4cff-bb0f-b6b0fca77438 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:20.939442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98753ee-91d7-4cff-bb0f-b6b0fca77438 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:20.942556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f98753ee-91d7-4cff-bb0f-b6b0fca77438 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:20.972657 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:21.065360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98753ee-91d7-4cff-bb0f-b6b0fca77438 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:36:21.067223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98753ee-91d7-4cff-bb0f-b6b0fca77438 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:36:21.097570 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 698/1363] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:36:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:36:23.559086 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65134fa-9c56-40cb-a4d0-4135ca86bd4d None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:23.559661 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65134fa-9c56-40cb-a4d0-4135ca86bd4d None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:23.559661 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65134fa-9c56-40cb-a4d0-4135ca86bd4d None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:23.560929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 666/1364] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:23.569859 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6175d4-2ad2-4cf5-a83d-2fc6916ac504 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:23.570386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6175d4-2ad2-4cf5-a83d-2fc6916ac504 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:23.570386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6175d4-2ad2-4cf5-a83d-2fc6916ac504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:23.648961 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-4f6175d4-2ad2-4cf5-a83d-2fc6916ac504 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:23.650285 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-4f6175d4-2ad2-4cf5-a83d-2fc6916ac504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:23.761308 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:23.764595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 699/1365] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:36:24.060736 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba5f0e-f21f-4931-bb17-9c0ebdc07192 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:24.061709 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba5f0e-f21f-4931-bb17-9c0ebdc07192 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:24.062047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba5f0e-f21f-4931-bb17-9c0ebdc07192 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:24.063472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 667/1366] 10.222.0.59 () {54 vars in 962 bytes} [Mon Aug 29 22:36:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:24.070967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905830e-ae30-4234-a7df-772504928c48 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:24.071365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905830e-ae30-4234-a7df-772504928c48 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:24.071694 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905830e-ae30-4234-a7df-772504928c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:24.163937 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-3905830e-ae30-4234-a7df-772504928c48 None None] MFA Rules not processed for user `7f21105679f245a19ea4850e6c010dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:24.165262 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3905830e-ae30-4234-a7df-772504928c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:24.296692 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 700/1367] 10.222.0.59 () {58 vars in 1059 bytes} [Mon Aug 29 22:36:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:36:24.307769 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba5f0e-f21f-4931-bb17-9c0ebdc07192 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:36:24.311156 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-daba5f0e-f21f-4931-bb17-9c0ebdc07192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:24.490415 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba5f0e-f21f-4931-bb17-9c0ebdc07192 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:36:24.511962 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:24.616311 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8385c-1a6c-4fb2-a203-91d994909e92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:36:24.617597 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8385c-1a6c-4fb2-a203-91d994909e92 service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:24.618038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8385c-1a6c-4fb2-a203-91d994909e92 service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:24.618446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8385c-1a6c-4fb2-a203-91d994909e92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:24.620255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ccc8385c-1a6c-4fb2-a203-91d994909e92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:24.715963 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8385c-1a6c-4fb2-a203-91d994909e92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:36:24.717835 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8385c-1a6c-4fb2-a203-91d994909e92 service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:36:24.738226 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 668/1368] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:36:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:36:25.231870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e06bf-0251-4b79-b458-22dd0ab0d682 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:25.233423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e06bf-0251-4b79-b458-22dd0ab0d682 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:25.233423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e06bf-0251-4b79-b458-22dd0ab0d682 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:25.234726 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 701/1369] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:36:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:25.240709 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f02fd-1063-4d15-b3e8-0eea6af684f5 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:25.240995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f02fd-1063-4d15-b3e8-0eea6af684f5 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:25.241268 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f02fd-1063-4d15-b3e8-0eea6af684f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:25.278288 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:25.322116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-d12f02fd-1063-4d15-b3e8-0eea6af684f5 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:25.323330 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d12f02fd-1063-4d15-b3e8-0eea6af684f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:25.444186 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 669/1370] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:36:26.020924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d272af0-4c88-403e-81b6-0d84fdbda450 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:26.021369 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d272af0-4c88-403e-81b6-0d84fdbda450 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:26.021742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d272af0-4c88-403e-81b6-0d84fdbda450 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:26.023432 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 702/1371] 10.222.0.59 () {54 vars in 964 bytes} [Mon Aug 29 22:36:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:26.029385 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7718308-8199-4ae0-81ad-d78a4ac4e68a None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:26.029909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7718308-8199-4ae0-81ad-d78a4ac4e68a None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:26.029909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7718308-8199-4ae0-81ad-d78a4ac4e68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:26.116512 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-a7718308-8199-4ae0-81ad-d78a4ac4e68a None None] MFA Rules not processed for user `92af56c889e640e7815c1641c05a9af3`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:26.117950 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a7718308-8199-4ae0-81ad-d78a4ac4e68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:26.171532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:26.207593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 670/1372] 10.222.0.59 () {58 vars in 1061 bytes} [Mon Aug 29 22:36:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:36:26.216393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d272af0-4c88-403e-81b6-0d84fdbda450 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:36:26.218147 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-1d272af0-4c88-403e-81b6-0d84fdbda450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:26.278172 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:26.338045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d272af0-4c88-403e-81b6-0d84fdbda450 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:36:26.432731 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed503a-22b0-491b-805d-2d623ffda04d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:36:26.433408 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed503a-22b0-491b-805d-2d623ffda04d service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:26.433753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed503a-22b0-491b-805d-2d623ffda04d service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:26.434090 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed503a-22b0-491b-805d-2d623ffda04d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:26.435763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-5eed503a-22b0-491b-805d-2d623ffda04d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:26.545002 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:26.564040 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed503a-22b0-491b-805d-2d623ffda04d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:36:26.567160 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed503a-22b0-491b-805d-2d623ffda04d service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:36:26.591464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 703/1373] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:36:26.999577 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0761626-1477-4519-9ed2-18f478f9859d None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:26.999778 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0761626-1477-4519-9ed2-18f478f9859d None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:27.000040 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0761626-1477-4519-9ed2-18f478f9859d None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:27.001465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 671/1374] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:27.007529 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936883f3-e696-49cd-89e5-55ec87a28f2d None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:27.007973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936883f3-e696-49cd-89e5-55ec87a28f2d None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:27.008547 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936883f3-e696-49cd-89e5-55ec87a28f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:27.093871 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-936883f3-e696-49cd-89e5-55ec87a28f2d None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:27.095100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-936883f3-e696-49cd-89e5-55ec87a28f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:27.213463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 704/1375] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:36:27.524807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0761626-1477-4519-9ed2-18f478f9859d None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:36:27.526298 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b0761626-1477-4519-9ed2-18f478f9859d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:27.681387 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0761626-1477-4519-9ed2-18f478f9859d None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:36:27.734439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:27.800927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae8dcf-2455-48d0-9dc3-b7228051e0dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:36:27.801501 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae8dcf-2455-48d0-9dc3-b7228051e0dc service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:27.801776 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae8dcf-2455-48d0-9dc3-b7228051e0dc service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:27.802043 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae8dcf-2455-48d0-9dc3-b7228051e0dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:27.803657 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-8fae8dcf-2455-48d0-9dc3-b7228051e0dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:27.900116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae8dcf-2455-48d0-9dc3-b7228051e0dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:36:27.901777 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae8dcf-2455-48d0-9dc3-b7228051e0dc service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:36:27.921202 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 672/1376] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:36:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:36:30.064223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a94981-9459-46a0-9516-47d9290cd643 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:30.064746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a94981-9459-46a0-9516-47d9290cd643 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:30.064746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a94981-9459-46a0-9516-47d9290cd643 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:30.066943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 705/1377] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:36:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:30.072502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f6365-eba4-4049-97fe-53a377749a96 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:30.072947 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f6365-eba4-4049-97fe-53a377749a96 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:30.073042 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f6365-eba4-4049-97fe-53a377749a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:30.149364 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-0a1f6365-eba4-4049-97fe-53a377749a96 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:30.150633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-0a1f6365-eba4-4049-97fe-53a377749a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:30.157443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:30.280473 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 673/1378] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:36:30.597403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a94981-9459-46a0-9516-47d9290cd643 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:36:30.599153 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-88a94981-9459-46a0-9516-47d9290cd643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:30.677709 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:30.773868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a94981-9459-46a0-9516-47d9290cd643 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:36:30.902451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664e0b6-974d-4e5a-8b1f-ec2b341b20de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:36:30.903246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664e0b6-974d-4e5a-8b1f-ec2b341b20de service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:30.903615 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664e0b6-974d-4e5a-8b1f-ec2b341b20de service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:30.903976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664e0b6-974d-4e5a-8b1f-ec2b341b20de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:30.905688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-d664e0b6-974d-4e5a-8b1f-ec2b341b20de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:30.938622 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:31.024290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664e0b6-974d-4e5a-8b1f-ec2b341b20de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:36:31.026088 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664e0b6-974d-4e5a-8b1f-ec2b341b20de service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:36:31.049238 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 706/1379] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:36:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:36:33.304616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a98c0-4f59-42f8-9f9f-b60a5cb46587 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:33.305360 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a98c0-4f59-42f8-9f9f-b60a5cb46587 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:33.305360 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a98c0-4f59-42f8-9f9f-b60a5cb46587 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:33.307515 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 674/1380] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:36:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:33.314427 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c584ef-1f31-4a4a-b74f-afe6013de202 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:33.315699 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c584ef-1f31-4a4a-b74f-afe6013de202 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:33.315964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c584ef-1f31-4a4a-b74f-afe6013de202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:33.400404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-46c584ef-1f31-4a4a-b74f-afe6013de202 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:33.402178 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-46c584ef-1f31-4a4a-b74f-afe6013de202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:33.539487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:33.552067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 707/1381] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:36:33.879117 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a98c0-4f59-42f8-9f9f-b60a5cb46587 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:36:33.881946 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-519a98c0-4f59-42f8-9f9f-b60a5cb46587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:34.047776 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:34.061382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a98c0-4f59-42f8-9f9f-b60a5cb46587 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:36:34.192071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf29bc5-0f2b-460c-b1d5-1c23de688809 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:36:34.192716 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf29bc5-0f2b-460c-b1d5-1c23de688809 service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:34.192992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf29bc5-0f2b-460c-b1d5-1c23de688809 service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:34.193261 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf29bc5-0f2b-460c-b1d5-1c23de688809 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:34.195146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-dbf29bc5-0f2b-460c-b1d5-1c23de688809 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:34.303758 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf29bc5-0f2b-460c-b1d5-1c23de688809 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:36:34.305376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf29bc5-0f2b-460c-b1d5-1c23de688809 service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:36:34.316200 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:34.325760 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 675/1382] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:36:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:36:36.494011 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d303dc-c563-47ca-a87e-790b7a9c4171 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:36.494562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d303dc-c563-47ca-a87e-790b7a9c4171 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:36.494747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d303dc-c563-47ca-a87e-790b7a9c4171 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:36.496534 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 708/1383] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:36:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:36.502749 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e8cd5-3917-4377-91dc-50da24c8e1eb None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:36.503200 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e8cd5-3917-4377-91dc-50da24c8e1eb None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:36.503719 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e8cd5-3917-4377-91dc-50da24c8e1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:36.575533 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-156e8cd5-3917-4377-91dc-50da24c8e1eb None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:36.576790 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-156e8cd5-3917-4377-91dc-50da24c8e1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:36.701831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 676/1384] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:36:36.996003 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d303dc-c563-47ca-a87e-790b7a9c4171 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:36:36.998525 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-98d303dc-c563-47ca-a87e-790b7a9c4171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:37.164191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d303dc-c563-47ca-a87e-790b7a9c4171 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:36:37.235988 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:37.291886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31134f3c-a2d9-44e5-b4d0-4eabecd106e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:36:37.292919 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31134f3c-a2d9-44e5-b4d0-4eabecd106e6 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:37.293264 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31134f3c-a2d9-44e5-b4d0-4eabecd106e6 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:37.293620 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31134f3c-a2d9-44e5-b4d0-4eabecd106e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:37.295544 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-31134f3c-a2d9-44e5-b4d0-4eabecd106e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:37.412479 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31134f3c-a2d9-44e5-b4d0-4eabecd106e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:36:37.414813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31134f3c-a2d9-44e5-b4d0-4eabecd106e6 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:36:37.443179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 709/1385] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:36:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:36:39.472350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be18ef-e7cb-48a6-b994-5c2234158f87 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:39.472885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be18ef-e7cb-48a6-b994-5c2234158f87 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:39.473118 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be18ef-e7cb-48a6-b994-5c2234158f87 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:39.474831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 677/1386] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:36:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:39.484161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bed364-15f0-449e-9bf3-ecad53f1bb61 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:39.484593 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bed364-15f0-449e-9bf3-ecad53f1bb61 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:39.484723 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bed364-15f0-449e-9bf3-ecad53f1bb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:39.572654 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-75bed364-15f0-449e-9bf3-ecad53f1bb61 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:39.574370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-75bed364-15f0-449e-9bf3-ecad53f1bb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:39.619639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:39.695894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 710/1387] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:36:40.075561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be18ef-e7cb-48a6-b994-5c2234158f87 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:36:40.077114 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-83be18ef-e7cb-48a6-b994-5c2234158f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:40.166793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:40.242850 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be18ef-e7cb-48a6-b994-5c2234158f87 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:36:40.366253 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de0713-c319-4786-b9ff-5c3de6b6aa9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:36:40.367127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de0713-c319-4786-b9ff-5c3de6b6aa9d service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:40.367601 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de0713-c319-4786-b9ff-5c3de6b6aa9d service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:40.368035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de0713-c319-4786-b9ff-5c3de6b6aa9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:40.370208 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-59de0713-c319-4786-b9ff-5c3de6b6aa9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:40.420902 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:40.477769 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de0713-c319-4786-b9ff-5c3de6b6aa9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:36:40.479383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de0713-c319-4786-b9ff-5c3de6b6aa9d service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:36:40.498483 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 678/1388] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:36:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:36:42.589223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0e7be-a7f1-44d1-9468-ffb49fa525e0 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:42.589786 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0e7be-a7f1-44d1-9468-ffb49fa525e0 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:42.589786 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0e7be-a7f1-44d1-9468-ffb49fa525e0 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:42.591130 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 711/1389] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:42.596697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54dd7b-38bc-44a4-90b7-430bb4eac948 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:42.597174 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54dd7b-38bc-44a4-90b7-430bb4eac948 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:42.597283 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54dd7b-38bc-44a4-90b7-430bb4eac948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:42.664239 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-2c54dd7b-38bc-44a4-90b7-430bb4eac948 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:42.665448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-2c54dd7b-38bc-44a4-90b7-430bb4eac948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:42.768300 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 679/1390] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:36:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:36:42.839899 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91ed9b-1811-4e4e-8344-840cc285a64e None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:42.840095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91ed9b-1811-4e4e-8344-840cc285a64e None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:42.840378 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91ed9b-1811-4e4e-8344-840cc285a64e None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:42.841728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 712/1391] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:42.848165 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9e58b-6ff9-4270-a89b-a003f31d2e0e None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:42.848443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9e58b-6ff9-4270-a89b-a003f31d2e0e None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:42.848715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9e58b-6ff9-4270-a89b-a003f31d2e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:42.864849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:42.920045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-33c9e58b-6ff9-4270-a89b-a003f31d2e0e None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:42.921121 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-33c9e58b-6ff9-4270-a89b-a003f31d2e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:43.025786 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 680/1392] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:36:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:36:43.066446 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91ed9b-1811-4e4e-8344-840cc285a64e None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:36:43.067971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-5d91ed9b-1811-4e4e-8344-840cc285a64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:43.222670 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91ed9b-1811-4e4e-8344-840cc285a64e None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:36:43.241394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:43.334274 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e8dcb-2a56-497d-8a4b-477469a1c736 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:43.334633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e8dcb-2a56-497d-8a4b-477469a1c736 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:43.335039 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e8dcb-2a56-497d-8a4b-477469a1c736 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:43.336531 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 681/1393] 10.222.0.59 () {54 vars in 967 bytes} [Mon Aug 29 22:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:36:43.341445 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fb16a-e148-4410-bd0c-04cc4279c25d None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:43.341724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fb16a-e148-4410-bd0c-04cc4279c25d None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:43.341991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fb16a-e148-4410-bd0c-04cc4279c25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:43.353355 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9909e31-5380-4b5b-975e-a92ebfd1b255 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:36:43.353964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9909e31-5380-4b5b-975e-a92ebfd1b255 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:43.354256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9909e31-5380-4b5b-975e-a92ebfd1b255 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:43.354503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9909e31-5380-4b5b-975e-a92ebfd1b255 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:43.356101 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b9909e31-5380-4b5b-975e-a92ebfd1b255 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:43.427009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-280fb16a-e148-4410-bd0c-04cc4279c25d None None] MFA Rules not processed for user `4db08aa2e4ef4478a3909a876754814c`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:43.428241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-280fb16a-e148-4410-bd0c-04cc4279c25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:43.452554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9909e31-5380-4b5b-975e-a92ebfd1b255 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:36:43.454163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9909e31-5380-4b5b-975e-a92ebfd1b255 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:36:43.464757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:43.475640 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 713/1394] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:36:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:36:43.510789 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 682/1395] 10.222.0.59 () {58 vars in 1064 bytes} [Mon Aug 29 22:36:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:36:43.518512 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9909e31-5380-4b5b-975e-a92ebfd1b255 service nova] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:36:43.519696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b9909e31-5380-4b5b-975e-a92ebfd1b255 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:43.541706 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:43.596882 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9909e31-5380-4b5b-975e-a92ebfd1b255 service nova] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:36:43.674721 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9aa009-3e8a-4730-a65b-121e2f0ac8f3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4db08aa2e4ef4478a3909a876754814c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:36:43.675412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9aa009-3e8a-4730-a65b-121e2f0ac8f3 service glance] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:43.675714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9aa009-3e8a-4730-a65b-121e2f0ac8f3 service glance] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:43.675997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9aa009-3e8a-4730-a65b-121e2f0ac8f3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:43.677582 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-2a9aa009-3e8a-4730-a65b-121e2f0ac8f3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:43.791202 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:43.806281 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9aa009-3e8a-4730-a65b-121e2f0ac8f3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:36:43.807842 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9aa009-3e8a-4730-a65b-121e2f0ac8f3 service glance] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:36:43.831620 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 714/1396] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:36:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:36:44.756583 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f6ba6-1531-47d9-9df1-16b848f1bb45 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:44.763082 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f6ba6-1531-47d9-9df1-16b848f1bb45 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:44.763082 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f6ba6-1531-47d9-9df1-16b848f1bb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:44.833300 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-268f6ba6-1531-47d9-9df1-16b848f1bb45 None None] MFA Rules not processed for user `980157efae8f43b59dc063fda04c0348`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:44.834506 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-268f6ba6-1531-47d9-9df1-16b848f1bb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:44.932153 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 683/1397] 10.222.0.59 () {58 vars in 1009 bytes} [Mon Aug 29 22:36:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:36:44.978808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1cbcc-94be-4a8e-a613-46107250acc8 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:44.979517 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1cbcc-94be-4a8e-a613-46107250acc8 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:44.979517 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1cbcc-94be-4a8e-a613-46107250acc8 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:44.981001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 715/1398] 10.222.0.59 () {54 vars in 943 bytes} [Mon Aug 29 22:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:44.989334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0decb0-2f64-4ad1-8c4f-f90d2ec73e93 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:44.989699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0decb0-2f64-4ad1-8c4f-f90d2ec73e93 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:44.990018 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0decb0-2f64-4ad1-8c4f-f90d2ec73e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:45.049450 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:45.069220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-5e0decb0-2f64-4ad1-8c4f-f90d2ec73e93 None None] MFA Rules not processed for user `f1f0503ae1b24ce6a8adb3cfd7973f14`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:45.070418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-5e0decb0-2f64-4ad1-8c4f-f90d2ec73e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:45.177941 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 684/1399] 10.222.0.59 () {58 vars in 1040 bytes} [Mon Aug 29 22:36:44 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:36:45.187223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1cbcc-94be-4a8e-a613-46107250acc8 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:36:45.188773 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-17c1cbcc-94be-4a8e-a613-46107250acc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:45.334697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1cbcc-94be-4a8e-a613-46107250acc8 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:36:45.418539 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:45.464759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a841b-c61a-477c-9b20-727d2be0d569 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f0503ae1b24ce6a8adb3cfd7973f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:36:45.465273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a841b-c61a-477c-9b20-727d2be0d569 service swift] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:45.465547 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a841b-c61a-477c-9b20-727d2be0d569 service swift] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:45.465813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a841b-c61a-477c-9b20-727d2be0d569 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:45.467434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-215a841b-c61a-477c-9b20-727d2be0d569 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:45.555232 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a841b-c61a-477c-9b20-727d2be0d569 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:36:45.556866 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a841b-c61a-477c-9b20-727d2be0d569 service swift] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:36:45.565611 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 716/1400] 10.222.0.59 () {60 vars in 1384 bytes} [Mon Aug 29 22:36:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 22:36:45.658995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db7c6b-d428-4745-8fea-47d28787c2d9 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:45.659599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db7c6b-d428-4745-8fea-47d28787c2d9 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:45.659900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db7c6b-d428-4745-8fea-47d28787c2d9 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:45.661462 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 685/1401] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:36:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:45.666762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43681d8b-2ea0-4622-8f13-e4e000f57f64 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:45.667430 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43681d8b-2ea0-4622-8f13-e4e000f57f64 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:45.667553 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43681d8b-2ea0-4622-8f13-e4e000f57f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:45.736825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-43681d8b-2ea0-4622-8f13-e4e000f57f64 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:45.737961 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-43681d8b-2ea0-4622-8f13-e4e000f57f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:45.794179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:45.841562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 717/1402] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:36:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:36:46.139165 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db7c6b-d428-4745-8fea-47d28787c2d9 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:36:46.140714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-94db7c6b-d428-4745-8fea-47d28787c2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:46.275269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:46.339874 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db7c6b-d428-4745-8fea-47d28787c2d9 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:36:46.496041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581ad30-cdc6-4fa8-941f-53f798e7d7aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:36:46.496923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581ad30-cdc6-4fa8-941f-53f798e7d7aa service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:46.497149 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581ad30-cdc6-4fa8-941f-53f798e7d7aa service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:46.497574 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581ad30-cdc6-4fa8-941f-53f798e7d7aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:46.499225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-0581ad30-cdc6-4fa8-941f-53f798e7d7aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:46.591823 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:46.613548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581ad30-cdc6-4fa8-941f-53f798e7d7aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:36:46.615476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581ad30-cdc6-4fa8-941f-53f798e7d7aa service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:36:46.635739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 686/1403] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:36:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:36:48.836455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b932f-0878-4830-af6a-2cb4d0252190 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:48.837543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b932f-0878-4830-af6a-2cb4d0252190 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:48.837824 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b932f-0878-4830-af6a-2cb4d0252190 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:48.843179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 718/1404] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:36:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:48.846970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ea21d-02d1-4c56-bf62-3e83bc395b69 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:48.847471 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ea21d-02d1-4c56-bf62-3e83bc395b69 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:48.847902 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ea21d-02d1-4c56-bf62-3e83bc395b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:48.981958 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-cc1ea21d-02d1-4c56-bf62-3e83bc395b69 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:48.983219 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-cc1ea21d-02d1-4c56-bf62-3e83bc395b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:49.104284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 687/1405] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:36:49.403302 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b932f-0878-4830-af6a-2cb4d0252190 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:36:49.405061 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-202b932f-0878-4830-af6a-2cb4d0252190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:49.573802 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b932f-0878-4830-af6a-2cb4d0252190 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:36:49.623885 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:49.718511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a2553-a74b-404c-93c0-ca190c574439 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:36:49.719034 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a2553-a74b-404c-93c0-ca190c574439 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:49.719313 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a2553-a74b-404c-93c0-ca190c574439 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:49.719585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a2553-a74b-404c-93c0-ca190c574439 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:49.721925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-0f4a2553-a74b-404c-93c0-ca190c574439 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:49.832900 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a2553-a74b-404c-93c0-ca190c574439 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:36:49.834879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a2553-a74b-404c-93c0-ca190c574439 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:36:49.855461 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 719/1406] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:36:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:36:52.009492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dc60b-383d-4ba4-bc90-d76829023ef5 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:52.010068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dc60b-383d-4ba4-bc90-d76829023ef5 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:52.010068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dc60b-383d-4ba4-bc90-d76829023ef5 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:52.011369 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 688/1407] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:36:52.016930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e8bc6-3311-4485-871e-459fbee90a24 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:52.018169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e8bc6-3311-4485-871e-459fbee90a24 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:52.018169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e8bc6-3311-4485-871e-459fbee90a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:52.075211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:52.091645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-3c3e8bc6-3311-4485-871e-459fbee90a24 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:36:52.092904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3c3e8bc6-3311-4485-871e-459fbee90a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:52.212271 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 720/1408] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:36:52.510893 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dc60b-383d-4ba4-bc90-d76829023ef5 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:36:52.513060 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a67dc60b-383d-4ba4-bc90-d76829023ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:52.574586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:52.692651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dc60b-383d-4ba4-bc90-d76829023ef5 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:36:52.841221 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60acb55-b40d-49b0-a281-15d779ddfe5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:36:52.842161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60acb55-b40d-49b0-a281-15d779ddfe5d service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:36:52.844535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60acb55-b40d-49b0-a281-15d779ddfe5d service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:36:52.844535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60acb55-b40d-49b0-a281-15d779ddfe5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:36:52.845606 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b60acb55-b40d-49b0-a281-15d779ddfe5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:36:52.857660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:36:52.992824 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60acb55-b40d-49b0-a281-15d779ddfe5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:36:52.995392 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60acb55-b40d-49b0-a281-15d779ddfe5d service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:36:53.023319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 689/1409] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:36:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:37:02.207224 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=46 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:03.020061 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=41 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:10.634376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6649d57-4c8a-453c-b740-82dbd9f0d360 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:10.635065 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6649d57-4c8a-453c-b740-82dbd9f0d360 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:10.635180 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6649d57-4c8a-453c-b740-82dbd9f0d360 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:10.636543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 721/1410] 10.222.0.59 () {54 vars in 918 bytes} [Mon Aug 29 22:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:37:10.642561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a78be-cced-46c1-8efe-aee23e348770 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:10.643425 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a78be-cced-46c1-8efe-aee23e348770 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:10.643425 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a78be-cced-46c1-8efe-aee23e348770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:10.726367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-4a3a78be-cced-46c1-8efe-aee23e348770 None None] MFA Rules not processed for user `7215ee41033742d79671f4f42b81cd06`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:37:10.727815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-4a3a78be-cced-46c1-8efe-aee23e348770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:10.847137 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 690/1411] 10.222.0.59 () {58 vars in 1015 bytes} [Mon Aug 29 22:37:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:37:10.868879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37bbb2-d993-427f-8bb2-c8ffb4af110d None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:10.869255 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37bbb2-d993-427f-8bb2-c8ffb4af110d None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:10.869696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37bbb2-d993-427f-8bb2-c8ffb4af110d None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:10.871206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 722/1412] 10.222.0.59 () {54 vars in 918 bytes} [Mon Aug 29 22:37:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:37:10.877116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502578ad-37f0-4599-ae16-18321b1346d0 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:10.877424 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502578ad-37f0-4599-ae16-18321b1346d0 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:10.877664 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502578ad-37f0-4599-ae16-18321b1346d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:10.973840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:10.979240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-502578ad-37f0-4599-ae16-18321b1346d0 None None] MFA Rules not processed for user `7f21105679f245a19ea4850e6c010dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:37:10.981343 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-502578ad-37f0-4599-ae16-18321b1346d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:11.151520 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 691/1413] 10.222.0.59 () {58 vars in 1015 bytes} [Mon Aug 29 22:37:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:37:13.708405 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56363fb0-eac9-4485-8e03-7aaab5f9e911 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:13.708990 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56363fb0-eac9-4485-8e03-7aaab5f9e911 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:13.708990 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56363fb0-eac9-4485-8e03-7aaab5f9e911 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:13.710593 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 723/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:37:13.716989 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258f20b-9cdf-443b-b53e-6a3b6154e8e6 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:13.717817 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258f20b-9cdf-443b-b53e-6a3b6154e8e6 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:13.717992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258f20b-9cdf-443b-b53e-6a3b6154e8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:13.801188 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-4258f20b-9cdf-443b-b53e-6a3b6154e8e6 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:37:13.802463 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-4258f20b-9cdf-443b-b53e-6a3b6154e8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:13.869883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:13.918038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 692/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:37:14.217376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56363fb0-eac9-4485-8e03-7aaab5f9e911 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:37:14.218978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-56363fb0-eac9-4485-8e03-7aaab5f9e911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:14.406126 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56363fb0-eac9-4485-8e03-7aaab5f9e911 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:37:14.512471 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:14.592185 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae0a5f-5af1-475f-8504-8b1bfb064488 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:37:14.593218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae0a5f-5af1-475f-8504-8b1bfb064488 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:14.593686 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae0a5f-5af1-475f-8504-8b1bfb064488 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:14.594687 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae0a5f-5af1-475f-8504-8b1bfb064488 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:14.597864 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-0aae0a5f-5af1-475f-8504-8b1bfb064488 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:14.768750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aae0a5f-5af1-475f-8504-8b1bfb064488 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:37:14.771610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aae0a5f-5af1-475f-8504-8b1bfb064488 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:37:14.807036 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 724/1416] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:37:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:37:15.793856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26906df-6149-4b55-8c79-26ff3a33ac33 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:15.794758 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26906df-6149-4b55-8c79-26ff3a33ac33 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:15.794758 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26906df-6149-4b55-8c79-26ff3a33ac33 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:15.796660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 693/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:37:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:37:15.802844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7deb4e0-f1fa-4768-bc26-859773cc011b None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:15.803487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7deb4e0-f1fa-4768-bc26-859773cc011b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:15.803487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7deb4e0-f1fa-4768-bc26-859773cc011b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:15.895824 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-b7deb4e0-f1fa-4768-bc26-859773cc011b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:37:15.897783 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b7deb4e0-f1fa-4768-bc26-859773cc011b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:15.927890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:16.009810 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 725/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:37:16.252536 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26906df-6149-4b55-8c79-26ff3a33ac33 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:37:16.254078 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-c26906df-6149-4b55-8c79-26ff3a33ac33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:16.441306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26906df-6149-4b55-8c79-26ff3a33ac33 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:37:16.523112 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:16.640436 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446b1e4-6627-4167-a9a1-db57da296b8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:37:16.642048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446b1e4-6627-4167-a9a1-db57da296b8b service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:16.643001 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446b1e4-6627-4167-a9a1-db57da296b8b service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:16.643452 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446b1e4-6627-4167-a9a1-db57da296b8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:16.646618 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-e446b1e4-6627-4167-a9a1-db57da296b8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:16.786462 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446b1e4-6627-4167-a9a1-db57da296b8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:37:16.788024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446b1e4-6627-4167-a9a1-db57da296b8b service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:37:16.820509 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 694/1419] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:37:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:37:17.825659 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25722d-c5f6-442a-838f-9d1d9b5aed6d None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:17.826213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25722d-c5f6-442a-838f-9d1d9b5aed6d None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:17.826213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25722d-c5f6-442a-838f-9d1d9b5aed6d None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:17.827976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 726/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:37:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:37:17.834926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723354f-97c1-41c9-abe6-94f0d73b81af None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:17.835492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723354f-97c1-41c9-abe6-94f0d73b81af None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:17.835492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723354f-97c1-41c9-abe6-94f0d73b81af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:17.907537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:17.933103 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-6723354f-97c1-41c9-abe6-94f0d73b81af None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:37:17.934361 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6723354f-97c1-41c9-abe6-94f0d73b81af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:18.051599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 695/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:37:18.214392 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25722d-c5f6-442a-838f-9d1d9b5aed6d None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:37:18.216337 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-cb25722d-c5f6-442a-838f-9d1d9b5aed6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:18.396383 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25722d-c5f6-442a-838f-9d1d9b5aed6d None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:37:18.412092 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:18.536543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d7e982-69da-4248-a539-7dcccaaa01da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:37:18.537056 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7e982-69da-4248-a539-7dcccaaa01da service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:18.537331 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7e982-69da-4248-a539-7dcccaaa01da service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:18.537598 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7e982-69da-4248-a539-7dcccaaa01da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:18.539371 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-03d7e982-69da-4248-a539-7dcccaaa01da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:18.646829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d7e982-69da-4248-a539-7dcccaaa01da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:37:18.648888 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d7e982-69da-4248-a539-7dcccaaa01da service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:37:18.672748 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 727/1422] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:37:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:37:19.597882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6bf8c-a318-494f-9826-7eaf442dddf5 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:19.598415 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6bf8c-a318-494f-9826-7eaf442dddf5 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:19.598591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6bf8c-a318-494f-9826-7eaf442dddf5 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:19.600038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 696/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:37:19.606214 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383fd6d-6358-496f-b2a5-26b6679e1245 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:19.606983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383fd6d-6358-496f-b2a5-26b6679e1245 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:19.607172 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383fd6d-6358-496f-b2a5-26b6679e1245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:19.629992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:19.705930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-7383fd6d-6358-496f-b2a5-26b6679e1245 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:37:19.707160 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-7383fd6d-6358-496f-b2a5-26b6679e1245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:19.819704 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 728/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:37:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:37:19.983109 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6bf8c-a318-494f-9826-7eaf442dddf5 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:37:19.985388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-c6d6bf8c-a318-494f-9826-7eaf442dddf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:20.164937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:20.206632 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6bf8c-a318-494f-9826-7eaf442dddf5 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:37:20.335126 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75ead2-82c5-45d3-90a9-76caf61ef04b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:37:20.335799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75ead2-82c5-45d3-90a9-76caf61ef04b service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:20.336114 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75ead2-82c5-45d3-90a9-76caf61ef04b service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:20.336454 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75ead2-82c5-45d3-90a9-76caf61ef04b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:20.338341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-8d75ead2-82c5-45d3-90a9-76caf61ef04b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:20.429984 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:20.445615 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75ead2-82c5-45d3-90a9-76caf61ef04b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:37:20.447582 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75ead2-82c5-45d3-90a9-76caf61ef04b service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:37:20.474048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 697/1425] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:37:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:37:21.462691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1aa485-131b-422c-b88b-380ab344dea8 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:21.463464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1aa485-131b-422c-b88b-380ab344dea8 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:21.463464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1aa485-131b-422c-b88b-380ab344dea8 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:21.464820 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 729/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:37:21.474089 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf689a-a988-4c06-be4c-77dee4ee5e94 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:21.474789 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf689a-a988-4c06-be4c-77dee4ee5e94 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:21.475295 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf689a-a988-4c06-be4c-77dee4ee5e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:21.582801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-3ccf689a-a988-4c06-be4c-77dee4ee5e94 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:37:21.584370 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3ccf689a-a988-4c06-be4c-77dee4ee5e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:21.715473 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 698/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:37:21.871535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1aa485-131b-422c-b88b-380ab344dea8 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:37:21.873230 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6d1aa485-131b-422c-b88b-380ab344dea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:21.980341 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:22.054821 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1aa485-131b-422c-b88b-380ab344dea8 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:37:22.198185 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89687e98-d8e8-48bc-bf7b-42f58cbd32aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:37:22.198676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89687e98-d8e8-48bc-bf7b-42f58cbd32aa service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:22.198954 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89687e98-d8e8-48bc-bf7b-42f58cbd32aa service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:22.199225 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89687e98-d8e8-48bc-bf7b-42f58cbd32aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:22.200909 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-89687e98-d8e8-48bc-bf7b-42f58cbd32aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:22.262445 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:22.314335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89687e98-d8e8-48bc-bf7b-42f58cbd32aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:37:22.315968 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89687e98-d8e8-48bc-bf7b-42f58cbd32aa service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:37:22.337849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 730/1428] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:37:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:37:23.315222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9067c2-ed9f-4db0-9019-2d85b580b790 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:23.316066 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9067c2-ed9f-4db0-9019-2d85b580b790 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:23.316066 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9067c2-ed9f-4db0-9019-2d85b580b790 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:23.317377 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 699/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:37:23.323666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88a74b-7412-4a1f-998d-e097745fd826 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:23.324284 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88a74b-7412-4a1f-998d-e097745fd826 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:23.324284 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88a74b-7412-4a1f-998d-e097745fd826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:23.418555 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-7b88a74b-7412-4a1f-998d-e097745fd826 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:37:23.419798 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-7b88a74b-7412-4a1f-998d-e097745fd826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:23.540599 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 731/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:37:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:37:23.704854 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9067c2-ed9f-4db0-9019-2d85b580b790 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:37:23.706362 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ff9067c2-ed9f-4db0-9019-2d85b580b790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:23.762815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:23.916915 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9067c2-ed9f-4db0-9019-2d85b580b790 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:37:24.053024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473269ad-e8e7-451f-8a5d-2f11b3b91917 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:37:24.053642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473269ad-e8e7-451f-8a5d-2f11b3b91917 service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:24.053919 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473269ad-e8e7-451f-8a5d-2f11b3b91917 service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:24.054189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473269ad-e8e7-451f-8a5d-2f11b3b91917 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:24.055846 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-473269ad-e8e7-451f-8a5d-2f11b3b91917 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:24.064400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:24.163691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473269ad-e8e7-451f-8a5d-2f11b3b91917 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:37:24.165451 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473269ad-e8e7-451f-8a5d-2f11b3b91917 service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:37:24.186108 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 700/1431] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:37:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:37:25.098481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0985e-571c-46a3-a997-4d221b08df95 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:25.099076 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0985e-571c-46a3-a997-4d221b08df95 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:25.099457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0985e-571c-46a3-a997-4d221b08df95 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:25.102215 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 732/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:37:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:37:25.107895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c48ab-2953-4af7-8a0f-cf039a15eb26 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:25.108807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c48ab-2953-4af7-8a0f-cf039a15eb26 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:25.108807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c48ab-2953-4af7-8a0f-cf039a15eb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:25.196425 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-6b9c48ab-2953-4af7-8a0f-cf039a15eb26 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:37:25.197677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6b9c48ab-2953-4af7-8a0f-cf039a15eb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:25.276395 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:25.314363 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 701/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:37:25.490045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0985e-571c-46a3-a997-4d221b08df95 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:37:25.492253 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-65b0985e-571c-46a3-a997-4d221b08df95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:25.511028 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:25.665957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0985e-571c-46a3-a997-4d221b08df95 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:37:25.770538 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:25.800545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1479a-dcfc-4bbc-aec6-eadba3aa7e40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:37:25.801328 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1479a-dcfc-4bbc-aec6-eadba3aa7e40 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:25.801702 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1479a-dcfc-4bbc-aec6-eadba3aa7e40 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:25.802076 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1479a-dcfc-4bbc-aec6-eadba3aa7e40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:25.804128 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-40b1479a-dcfc-4bbc-aec6-eadba3aa7e40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:25.961005 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1479a-dcfc-4bbc-aec6-eadba3aa7e40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:37:25.962682 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1479a-dcfc-4bbc-aec6-eadba3aa7e40 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:37:25.985247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 733/1434] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:37:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:37:26.988510 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a2f70-7a21-445d-b1cf-39f725fe2c90 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:26.989258 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a2f70-7a21-445d-b1cf-39f725fe2c90 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:26.989258 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a2f70-7a21-445d-b1cf-39f725fe2c90 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:26.990631 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 702/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:37:26.996467 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269cefc-7463-47db-8400-3268852ff3cd None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:26.997325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269cefc-7463-47db-8400-3268852ff3cd None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:26.997325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269cefc-7463-47db-8400-3268852ff3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:27.089108 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-7269cefc-7463-47db-8400-3268852ff3cd None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:37:27.090346 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-7269cefc-7463-47db-8400-3268852ff3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:27.139905 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:27.209068 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 734/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:37:27.499950 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a2f70-7a21-445d-b1cf-39f725fe2c90 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:37:27.501040 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-062a2f70-7a21-445d-b1cf-39f725fe2c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:27.727718 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a2f70-7a21-445d-b1cf-39f725fe2c90 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:37:27.793888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:27.865757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39278386-f149-43db-95d1-99367b54fc9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:37:27.866380 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39278386-f149-43db-95d1-99367b54fc9c service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:27.866656 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39278386-f149-43db-95d1-99367b54fc9c service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:27.866925 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39278386-f149-43db-95d1-99367b54fc9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:27.868655 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-39278386-f149-43db-95d1-99367b54fc9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:27.981895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39278386-f149-43db-95d1-99367b54fc9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:37:27.983592 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39278386-f149-43db-95d1-99367b54fc9c service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:37:28.007071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 703/1437] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:37:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:37:29.475404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705e9f9-1c90-4f51-a1fe-35753cb2f812 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:29.476254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705e9f9-1c90-4f51-a1fe-35753cb2f812 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:29.476254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705e9f9-1c90-4f51-a1fe-35753cb2f812 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:29.477437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 735/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:37:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:37:29.485658 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a4127-24f2-4372-a772-5c0a0b774692 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:29.486485 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a4127-24f2-4372-a772-5c0a0b774692 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:29.486485 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a4127-24f2-4372-a772-5c0a0b774692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:29.595114 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-b70a4127-24f2-4372-a772-5c0a0b774692 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:37:29.597127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b70a4127-24f2-4372-a772-5c0a0b774692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:29.601158 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:29.722608 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 704/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:37:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:37:29.882284 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705e9f9-1c90-4f51-a1fe-35753cb2f812 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:37:29.883823 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-d705e9f9-1c90-4f51-a1fe-35753cb2f812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:30.056732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705e9f9-1c90-4f51-a1fe-35753cb2f812 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:37:30.092053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:30.209484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62c169-3dd8-4c1c-8a50-efad324d2d84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:37:30.210007 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62c169-3dd8-4c1c-8a50-efad324d2d84 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:30.210295 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62c169-3dd8-4c1c-8a50-efad324d2d84 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:30.210566 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62c169-3dd8-4c1c-8a50-efad324d2d84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:30.212486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3f62c169-3dd8-4c1c-8a50-efad324d2d84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:30.319665 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62c169-3dd8-4c1c-8a50-efad324d2d84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:37:30.321355 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62c169-3dd8-4c1c-8a50-efad324d2d84 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:37:30.344020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 736/1440] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:37:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:37:32.293271 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94a83a-e5ec-41ce-9912-093da6cada42 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:32.294123 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94a83a-e5ec-41ce-9912-093da6cada42 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:32.294123 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94a83a-e5ec-41ce-9912-093da6cada42 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:32.295783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 705/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:37:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:37:32.302417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6ef58-74fd-4582-b209-c2720cb55cf5 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:32.302961 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6ef58-74fd-4582-b209-c2720cb55cf5 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:32.303129 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6ef58-74fd-4582-b209-c2720cb55cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:32.350656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:32.380580 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-8ed6ef58-74fd-4582-b209-c2720cb55cf5 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:37:32.382789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8ed6ef58-74fd-4582-b209-c2720cb55cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:32.501894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 737/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:37:32.834285 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94a83a-e5ec-41ce-9912-093da6cada42 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:37:32.835838 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-7c94a83a-e5ec-41ce-9912-093da6cada42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:33.000482 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94a83a-e5ec-41ce-9912-093da6cada42 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:37:33.049217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:33.116121 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e5e0a-30d9-4360-b89f-e94771f035ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:37:33.116694 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e5e0a-30d9-4360-b89f-e94771f035ce service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:33.117019 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e5e0a-30d9-4360-b89f-e94771f035ce service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:33.117290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e5e0a-30d9-4360-b89f-e94771f035ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:33.118979 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-e66e5e0a-30d9-4360-b89f-e94771f035ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:33.263863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e5e0a-30d9-4360-b89f-e94771f035ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:37:33.265579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e5e0a-30d9-4360-b89f-e94771f035ce service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:37:33.292663 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 706/1443] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:37:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:37:35.793879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb1253-13fc-4bb2-9796-7e14a46e4ef3 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:35.794766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb1253-13fc-4bb2-9796-7e14a46e4ef3 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:35.794766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb1253-13fc-4bb2-9796-7e14a46e4ef3 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:35.796020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 738/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:37:35.802964 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2072d37-ebc7-46b8-a063-c7bb36fa6e9d None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:35.803548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2072d37-ebc7-46b8-a063-c7bb36fa6e9d None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:35.803548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2072d37-ebc7-46b8-a063-c7bb36fa6e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:35.883495 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-f2072d37-ebc7-46b8-a063-c7bb36fa6e9d None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:37:35.884864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f2072d37-ebc7-46b8-a063-c7bb36fa6e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:35.900594 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:36.024314 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 707/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:37:36.316010 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb1253-13fc-4bb2-9796-7e14a46e4ef3 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:37:36.317603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-96bb1253-13fc-4bb2-9796-7e14a46e4ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:36.440228 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb1253-13fc-4bb2-9796-7e14a46e4ef3 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:37:36.455142 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:36.526362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1e655-f8f3-49ad-9452-a0df48e92c18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:37:36.526975 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1e655-f8f3-49ad-9452-a0df48e92c18 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:36.527300 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1e655-f8f3-49ad-9452-a0df48e92c18 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:36.527632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1e655-f8f3-49ad-9452-a0df48e92c18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:36.529491 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b9b1e655-f8f3-49ad-9452-a0df48e92c18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:36.670808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1e655-f8f3-49ad-9452-a0df48e92c18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:37:36.672901 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1e655-f8f3-49ad-9452-a0df48e92c18 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:37:36.695661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 739/1446] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:37:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:37:39.214944 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fbf46-c407-4c41-a04c-478b79b197fa None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:39.215559 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fbf46-c407-4c41-a04c-478b79b197fa None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:39.215559 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fbf46-c407-4c41-a04c-478b79b197fa None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:39.216870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 708/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:37:39.223800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e326f6-9c55-48c2-8f45-f56a5e5fbad4 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:39.224684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e326f6-9c55-48c2-8f45-f56a5e5fbad4 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:39.224684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e326f6-9c55-48c2-8f45-f56a5e5fbad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:39.298567 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:39.334190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-38e326f6-9c55-48c2-8f45-f56a5e5fbad4 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:37:39.336258 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-38e326f6-9c55-48c2-8f45-f56a5e5fbad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:39.463682 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 740/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:37:39.762095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fbf46-c407-4c41-a04c-478b79b197fa None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:37:39.765705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-e13fbf46-c407-4c41-a04c-478b79b197fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:39.925234 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fbf46-c407-4c41-a04c-478b79b197fa None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:37:39.978724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:40.018552 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba208d-be71-4914-9452-a2c9fda4ecfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:37:40.019075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba208d-be71-4914-9452-a2c9fda4ecfe service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:40.019352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba208d-be71-4914-9452-a2c9fda4ecfe service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:40.019624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba208d-be71-4914-9452-a2c9fda4ecfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:40.021324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-48ba208d-be71-4914-9452-a2c9fda4ecfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:40.172827 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba208d-be71-4914-9452-a2c9fda4ecfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:37:40.174300 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba208d-be71-4914-9452-a2c9fda4ecfe service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:37:40.197763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 709/1449] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:37:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:37:42.367637 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a265d7-b151-42c9-9361-7456e775d434 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:42.368422 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a265d7-b151-42c9-9361-7456e775d434 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:42.368422 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a265d7-b151-42c9-9361-7456e775d434 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:42.369565 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 741/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:37:42.376257 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c0722-ea48-4293-aee6-fc4b49f068fd None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:42.377079 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c0722-ea48-4293-aee6-fc4b49f068fd None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:42.377079 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c0722-ea48-4293-aee6-fc4b49f068fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:42.464165 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-667c0722-ea48-4293-aee6-fc4b49f068fd None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:37:42.465467 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-667c0722-ea48-4293-aee6-fc4b49f068fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:42.492102 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:42.584718 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 710/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:37:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:37:42.881186 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a265d7-b151-42c9-9361-7456e775d434 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:37:42.882987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a8a265d7-b151-42c9-9361-7456e775d434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:43.042808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a265d7-b151-42c9-9361-7456e775d434 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:37:43.067955 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:43.142596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb32f1-1cd9-429a-81b4-f3f94214ce27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:37:43.143125 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb32f1-1cd9-429a-81b4-f3f94214ce27 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:43.143402 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb32f1-1cd9-429a-81b4-f3f94214ce27 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:43.143673 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb32f1-1cd9-429a-81b4-f3f94214ce27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:43.145393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-98bb32f1-1cd9-429a-81b4-f3f94214ce27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:43.300737 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bb32f1-1cd9-429a-81b4-f3f94214ce27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:37:43.302193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bb32f1-1cd9-429a-81b4-f3f94214ce27 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:37:43.323985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 742/1452] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:37:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:37:45.540546 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bec275-1915-46cf-9e5e-a7792f95ddb0 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:45.541127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bec275-1915-46cf-9e5e-a7792f95ddb0 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:45.541127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bec275-1915-46cf-9e5e-a7792f95ddb0 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:45.542586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 711/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:37:45.549310 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f31e3-cb2b-4ffd-83ad-d85bf6666379 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:45.549834 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f31e3-cb2b-4ffd-83ad-d85bf6666379 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:45.549834 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f31e3-cb2b-4ffd-83ad-d85bf6666379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:45.622554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:45.626402 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-164f31e3-cb2b-4ffd-83ad-d85bf6666379 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:37:45.627632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-164f31e3-cb2b-4ffd-83ad-d85bf6666379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:45.741289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 743/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:37:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:37:46.041281 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bec275-1915-46cf-9e5e-a7792f95ddb0 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:37:46.042835 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d6bec275-1915-46cf-9e5e-a7792f95ddb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:46.161351 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bec275-1915-46cf-9e5e-a7792f95ddb0 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:37:46.217266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:46.245176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b8592-cf24-4b73-83ed-75da15c4ade4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:37:46.245922 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b8592-cf24-4b73-83ed-75da15c4ade4 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:46.246286 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b8592-cf24-4b73-83ed-75da15c4ade4 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:46.246656 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b8592-cf24-4b73-83ed-75da15c4ade4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:46.248427 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-668b8592-cf24-4b73-83ed-75da15c4ade4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:46.372566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b8592-cf24-4b73-83ed-75da15c4ade4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:37:46.374329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b8592-cf24-4b73-83ed-75da15c4ade4 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:37:46.395152 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 712/1455] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:37:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:37:48.507052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62589ddf-6571-4607-a8e2-8fa944c9cc8f None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:48.507921 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62589ddf-6571-4607-a8e2-8fa944c9cc8f None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:48.508108 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62589ddf-6571-4607-a8e2-8fa944c9cc8f None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:48.509861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 744/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:37:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:37:48.516694 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ceba6b-4a83-4575-9978-1dcba769085f None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:48.517251 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ceba6b-4a83-4575-9978-1dcba769085f None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:48.517251 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ceba6b-4a83-4575-9978-1dcba769085f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:48.599422 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-d1ceba6b-4a83-4575-9978-1dcba769085f None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:37:48.600692 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d1ceba6b-4a83-4575-9978-1dcba769085f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:48.635144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:48.725975 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 713/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:37:49.031637 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62589ddf-6571-4607-a8e2-8fa944c9cc8f None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:37:49.033644 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-62589ddf-6571-4607-a8e2-8fa944c9cc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:49.154338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62589ddf-6571-4607-a8e2-8fa944c9cc8f None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:37:49.188032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:49.243002 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7144729-ccbd-471d-b19d-5e94d470aa86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:37:49.243823 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7144729-ccbd-471d-b19d-5e94d470aa86 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:49.244218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7144729-ccbd-471d-b19d-5e94d470aa86 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:49.244535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7144729-ccbd-471d-b19d-5e94d470aa86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:49.246574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e7144729-ccbd-471d-b19d-5e94d470aa86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:49.377062 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7144729-ccbd-471d-b19d-5e94d470aa86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:37:49.378580 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7144729-ccbd-471d-b19d-5e94d470aa86 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:37:49.401321 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 745/1458] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:37:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:37:51.527212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b7a6c-72cf-4c92-ab27-bded80523fd5 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:51.527891 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b7a6c-72cf-4c92-ab27-bded80523fd5 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:51.527891 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b7a6c-72cf-4c92-ab27-bded80523fd5 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:51.529480 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 714/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:37:51.535894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7d21c-6058-422c-b01d-171ddf107fae None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:51.536562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7d21c-6058-422c-b01d-171ddf107fae None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:51.536562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7d21c-6058-422c-b01d-171ddf107fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:51.616573 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-cfe7d21c-6058-422c-b01d-171ddf107fae None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:37:51.617768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-cfe7d21c-6058-422c-b01d-171ddf107fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:51.628425 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:51.777140 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 746/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:37:52.068269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b7a6c-72cf-4c92-ab27-bded80523fd5 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:37:52.069924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-534b7a6c-72cf-4c92-ab27-bded80523fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:52.192777 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b7a6c-72cf-4c92-ab27-bded80523fd5 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:37:52.257466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:52.279304 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6534f1-d3f4-4d35-95ec-1f9bd5cc894f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:37:52.280610 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6534f1-d3f4-4d35-95ec-1f9bd5cc894f service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:52.280877 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6534f1-d3f4-4d35-95ec-1f9bd5cc894f service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:52.281138 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6534f1-d3f4-4d35-95ec-1f9bd5cc894f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:52.282791 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-4c6534f1-d3f4-4d35-95ec-1f9bd5cc894f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:52.472941 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6534f1-d3f4-4d35-95ec-1f9bd5cc894f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:37:52.475937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6534f1-d3f4-4d35-95ec-1f9bd5cc894f service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:37:52.516777 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 715/1461] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:37:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:37:54.650197 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea6352-c8cb-4172-afdc-5f7bdaa06b6b None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:54.651124 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea6352-c8cb-4172-afdc-5f7bdaa06b6b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:54.651463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea6352-c8cb-4172-afdc-5f7bdaa06b6b None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:54.653003 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 747/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:37:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:37:54.660252 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57105730-e431-42f4-aab3-87283f42c786 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:54.661073 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57105730-e431-42f4-aab3-87283f42c786 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:54.661246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57105730-e431-42f4-aab3-87283f42c786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:54.741481 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-57105730-e431-42f4-aab3-87283f42c786 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:37:54.742893 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-57105730-e431-42f4-aab3-87283f42c786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:54.794131 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:54.864395 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 716/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:37:55.149844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea6352-c8cb-4172-afdc-5f7bdaa06b6b None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:37:55.151690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-50ea6352-c8cb-4172-afdc-5f7bdaa06b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:55.278190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea6352-c8cb-4172-afdc-5f7bdaa06b6b None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:37:55.323932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:55.365305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24393f3-27c8-4c50-9a86-fe97e7a04b84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:37:55.365924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24393f3-27c8-4c50-9a86-fe97e7a04b84 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:55.366261 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24393f3-27c8-4c50-9a86-fe97e7a04b84 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:55.366612 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24393f3-27c8-4c50-9a86-fe97e7a04b84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:55.368400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a24393f3-27c8-4c50-9a86-fe97e7a04b84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:55.497480 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24393f3-27c8-4c50-9a86-fe97e7a04b84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:37:55.499475 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24393f3-27c8-4c50-9a86-fe97e7a04b84 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:37:55.521324 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 748/1464] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:37:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:37:57.708368 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55e137-b3f0-4f0d-b963-4cd271a5317b None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:57.709243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55e137-b3f0-4f0d-b963-4cd271a5317b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:57.709243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55e137-b3f0-4f0d-b963-4cd271a5317b None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:57.710809 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 717/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:37:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:37:57.718870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd93fc-e670-4b2f-b4d2-933d558df8f2 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:57.719717 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd93fc-e670-4b2f-b4d2-933d558df8f2 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:57.719987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd93fc-e670-4b2f-b4d2-933d558df8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:57.807522 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-dbbd93fc-e670-4b2f-b4d2-933d558df8f2 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:37:57.808825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-dbbd93fc-e670-4b2f-b4d2-933d558df8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:57.833012 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:57.939040 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 749/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:37:58.235530 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55e137-b3f0-4f0d-b963-4cd271a5317b None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:37:58.237136 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-0b55e137-b3f0-4f0d-b963-4cd271a5317b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:58.359593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55e137-b3f0-4f0d-b963-4cd271a5317b None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:37:58.449814 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:37:58.458298 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b68827-7c81-42e6-925d-2d3058c7e1b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:37:58.468534 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b68827-7c81-42e6-925d-2d3058c7e1b4 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:37:58.468534 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b68827-7c81-42e6-925d-2d3058c7e1b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:37:58.468534 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b68827-7c81-42e6-925d-2d3058c7e1b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:37:58.468534 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d8b68827-7c81-42e6-925d-2d3058c7e1b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:37:58.636333 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b68827-7c81-42e6-925d-2d3058c7e1b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:37:58.640306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b68827-7c81-42e6-925d-2d3058c7e1b4 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:37:58.676004 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 718/1467] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:37:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:38:00.719181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa85bf-7236-4077-b06d-cd151d20f864 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:00.719835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa85bf-7236-4077-b06d-cd151d20f864 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:00.719835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa85bf-7236-4077-b06d-cd151d20f864 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:00.721160 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 750/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:38:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:38:00.732336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b9545-e821-464c-b3fc-5b16c024fe0c None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:00.733091 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b9545-e821-464c-b3fc-5b16c024fe0c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:00.733205 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b9545-e821-464c-b3fc-5b16c024fe0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:00.815621 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-6c3b9545-e821-464c-b3fc-5b16c024fe0c None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:38:00.816904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6c3b9545-e821-464c-b3fc-5b16c024fe0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:00.884491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:38:00.943934 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 719/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:38:01.230853 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa85bf-7236-4077-b06d-cd151d20f864 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:38:01.233342 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-91aa85bf-7236-4077-b06d-cd151d20f864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:01.391309 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa85bf-7236-4077-b06d-cd151d20f864 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:38:01.445273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:38:01.473117 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030ea6a-555b-42f9-a07b-b6aec61ff794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:38:01.474043 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030ea6a-555b-42f9-a07b-b6aec61ff794 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:01.474269 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030ea6a-555b-42f9-a07b-b6aec61ff794 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:01.474541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030ea6a-555b-42f9-a07b-b6aec61ff794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:01.476968 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a030ea6a-555b-42f9-a07b-b6aec61ff794 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:01.614996 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030ea6a-555b-42f9-a07b-b6aec61ff794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:38:01.616473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030ea6a-555b-42f9-a07b-b6aec61ff794 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:38:01.636860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 751/1470] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:38:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:38:10.942008 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=22 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:38:11.634405 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=66 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:38:22.664283 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a1ead-5838-4f5d-9491-2b16b7dd0765 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:22.665115 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a1ead-5838-4f5d-9491-2b16b7dd0765 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:22.665115 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a1ead-5838-4f5d-9491-2b16b7dd0765 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:22.666163 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 720/1471] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:38:22.672643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f31a2-22e6-4bbf-b635-d5265cc6810b None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:22.673435 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f31a2-22e6-4bbf-b635-d5265cc6810b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:22.673613 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f31a2-22e6-4bbf-b635-d5265cc6810b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:22.763292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-e52f31a2-22e6-4bbf-b635-d5265cc6810b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:38:22.764969 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e52f31a2-22e6-4bbf-b635-d5265cc6810b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:22.945082 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 752/1472] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:38:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:38:23.514744 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c376c73-f837-4155-9f93-70902c2a813c None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:23.515224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c376c73-f837-4155-9f93-70902c2a813c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:23.515704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c376c73-f837-4155-9f93-70902c2a813c None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:23.518197 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 721/1473] 10.222.0.59 () {54 vars in 967 bytes} [Mon Aug 29 22:38:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:38:23.524125 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d47d6-5f88-4d3e-83cb-d2ecddc77c14 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:23.524519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d47d6-5f88-4d3e-83cb-d2ecddc77c14 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:23.524835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d47d6-5f88-4d3e-83cb-d2ecddc77c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:23.632225 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-1f3d47d6-5f88-4d3e-83cb-d2ecddc77c14 None None] MFA Rules not processed for user `4db08aa2e4ef4478a3909a876754814c`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:38:23.634244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-1f3d47d6-5f88-4d3e-83cb-d2ecddc77c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:23.672444 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:38:23.748440 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 753/1474] 10.222.0.59 () {58 vars in 1064 bytes} [Mon Aug 29 22:38:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:38:23.756696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c376c73-f837-4155-9f93-70902c2a813c None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:38:23.758724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-5c376c73-f837-4155-9f93-70902c2a813c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:23.935950 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c376c73-f837-4155-9f93-70902c2a813c None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:38:24.067274 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f2880-8942-45e0-ba9c-34d1eae4bb88 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4db08aa2e4ef4478a3909a876754814c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:38:24.067882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f2880-8942-45e0-ba9c-34d1eae4bb88 service glance] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:24.068176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f2880-8942-45e0-ba9c-34d1eae4bb88 service glance] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:24.068452 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f2880-8942-45e0-ba9c-34d1eae4bb88 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:24.070125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-864f2880-8942-45e0-ba9c-34d1eae4bb88 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:24.133978 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:38:24.206706 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f2880-8942-45e0-ba9c-34d1eae4bb88 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:38:24.207943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f2880-8942-45e0-ba9c-34d1eae4bb88 service glance] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:38:24.229606 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 722/1475] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:38:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:38:26.462208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256069b-46e1-40d9-b6f6-1fae3ee675d0 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:26.462797 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256069b-46e1-40d9-b6f6-1fae3ee675d0 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:26.462797 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256069b-46e1-40d9-b6f6-1fae3ee675d0 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:26.464539 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 754/1476] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:38:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:38:26.471531 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd638b1-2224-4ed9-ab8b-ae813d09b725 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:26.472102 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd638b1-2224-4ed9-ab8b-ae813d09b725 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:26.472102 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd638b1-2224-4ed9-ab8b-ae813d09b725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:26.560810 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-fbd638b1-2224-4ed9-ab8b-ae813d09b725 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:38:26.562160 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-fbd638b1-2224-4ed9-ab8b-ae813d09b725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:26.674684 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 723/1477] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:38:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:38:26.953694 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256069b-46e1-40d9-b6f6-1fae3ee675d0 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:38:26.955455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b256069b-46e1-40d9-b6f6-1fae3ee675d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:27.102128 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256069b-46e1-40d9-b6f6-1fae3ee675d0 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:38:27.160038 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:38:27.214880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b913f4-85a1-418b-849b-6e09f1b79b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:38:27.216191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b913f4-85a1-418b-849b-6e09f1b79b54 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:27.216466 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b913f4-85a1-418b-849b-6e09f1b79b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:27.216827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b913f4-85a1-418b-849b-6e09f1b79b54 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:27.217479 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b913f4-85a1-418b-849b-6e09f1b79b54 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:38:27.219292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: WARNING py.warnings [None req-75b913f4-85a1-418b-849b-6e09f1b79b54 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:38:27.219292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: warnings.warn(msg) Aug 29 22:38:27.219292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: Aug 29 22:38:27.220484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b913f4-85a1-418b-849b-6e09f1b79b54 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:38:27.242437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 755/1478] 10.222.0.59 () {56 vars in 1108 bytes} [Mon Aug 29 22:38:26 2022] GET /identity/v3/projects => generated 2553 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:38:29.278703 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e912762-ac61-408f-a040-21f6de45d2fc None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:29.279340 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e912762-ac61-408f-a040-21f6de45d2fc None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:29.279340 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e912762-ac61-408f-a040-21f6de45d2fc None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:29.280721 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 724/1479] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:38:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:38:29.286737 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6352c5-43f6-442e-8b7e-4631a2f802c5 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:29.287419 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6352c5-43f6-442e-8b7e-4631a2f802c5 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:29.287419 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6352c5-43f6-442e-8b7e-4631a2f802c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:29.360952 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-cb6352c5-43f6-442e-8b7e-4631a2f802c5 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:38:29.362300 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-cb6352c5-43f6-442e-8b7e-4631a2f802c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:29.472233 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 756/1480] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:38:29.767840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e912762-ac61-408f-a040-21f6de45d2fc None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:38:29.769388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-4e912762-ac61-408f-a040-21f6de45d2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:29.779960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:38:29.944607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e912762-ac61-408f-a040-21f6de45d2fc None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:38:30.040711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:38:30.083024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475d2a8-3351-402c-86ef-a2078ef8920a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:38:30.083655 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475d2a8-3351-402c-86ef-a2078ef8920a service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:30.083929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475d2a8-3351-402c-86ef-a2078ef8920a service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:30.084237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475d2a8-3351-402c-86ef-a2078ef8920a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:30.086025 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-8475d2a8-3351-402c-86ef-a2078ef8920a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:30.204225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8475d2a8-3351-402c-86ef-a2078ef8920a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:38:30.207285 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8475d2a8-3351-402c-86ef-a2078ef8920a service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:38:30.241910 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 725/1481] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:38:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:38:32.583865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2067c3-a508-47f9-b250-be32f3e6d81e None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:32.584577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2067c3-a508-47f9-b250-be32f3e6d81e None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:32.584577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2067c3-a508-47f9-b250-be32f3e6d81e None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:32.585895 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 757/1482] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:38:32.593730 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b26b8-70b4-47f8-a1ad-974bef798b67 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:32.594330 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b26b8-70b4-47f8-a1ad-974bef798b67 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:32.594694 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b26b8-70b4-47f8-a1ad-974bef798b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:32.690113 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-220b26b8-70b4-47f8-a1ad-974bef798b67 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:38:32.691415 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-220b26b8-70b4-47f8-a1ad-974bef798b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:32.760193 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:38:32.820978 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 726/1483] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:38:33.146994 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2067c3-a508-47f9-b250-be32f3e6d81e None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:38:33.148619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-5f2067c3-a508-47f9-b250-be32f3e6d81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:33.176387 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:38:33.308463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2067c3-a508-47f9-b250-be32f3e6d81e None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:38:33.415309 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1258216-3ccc-4d2a-ab6d-f0585c550325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:38:33.415987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1258216-3ccc-4d2a-ab6d-f0585c550325 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:33.416298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1258216-3ccc-4d2a-ab6d-f0585c550325 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:33.416792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1258216-3ccc-4d2a-ab6d-f0585c550325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:33.420255 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-d1258216-3ccc-4d2a-ab6d-f0585c550325 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:33.515112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:38:33.560614 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1258216-3ccc-4d2a-ab6d-f0585c550325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:38:33.562079 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1258216-3ccc-4d2a-ab6d-f0585c550325 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:38:33.583603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 758/1484] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:38:36.046870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34104c6b-1c59-4120-a724-6ee5bfeaa664 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:36.047680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34104c6b-1c59-4120-a724-6ee5bfeaa664 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:36.047680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34104c6b-1c59-4120-a724-6ee5bfeaa664 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:36.048871 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 727/1485] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:38:36.055061 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388c125-fc84-48de-8a9a-7ebbae749dc5 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:36.063959 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388c125-fc84-48de-8a9a-7ebbae749dc5 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:36.063959 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388c125-fc84-48de-8a9a-7ebbae749dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:36.143503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-0388c125-fc84-48de-8a9a-7ebbae749dc5 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:38:36.144888 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-0388c125-fc84-48de-8a9a-7ebbae749dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:36.269743 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 759/1486] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:38:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:38:36.580884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34104c6b-1c59-4120-a724-6ee5bfeaa664 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:38:36.582496 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-34104c6b-1c59-4120-a724-6ee5bfeaa664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:36.705517 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34104c6b-1c59-4120-a724-6ee5bfeaa664 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:38:36.791530 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:38:36.796041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5a815-6207-49cc-bf38-6cf173523d86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:38:36.796766 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5a815-6207-49cc-bf38-6cf173523d86 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:36.797085 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5a815-6207-49cc-bf38-6cf173523d86 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:36.797397 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5a815-6207-49cc-bf38-6cf173523d86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:36.799270 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-28e5a815-6207-49cc-bf38-6cf173523d86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:37.078382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5a815-6207-49cc-bf38-6cf173523d86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:38:37.081146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5a815-6207-49cc-bf38-6cf173523d86 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:38:37.116973 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 728/1487] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:38:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:38:39.384673 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408dbde-41fe-40dc-b1b2-6ed4ec2dbf28 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:39.385256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408dbde-41fe-40dc-b1b2-6ed4ec2dbf28 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:39.385631 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408dbde-41fe-40dc-b1b2-6ed4ec2dbf28 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:39.388269 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 760/1488] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:38:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:38:39.394820 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454bc33-ff5d-4c9e-9b5b-98c2a8a93e5e None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:39.395440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454bc33-ff5d-4c9e-9b5b-98c2a8a93e5e None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:39.395440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454bc33-ff5d-4c9e-9b5b-98c2a8a93e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:39.476376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-9454bc33-ff5d-4c9e-9b5b-98c2a8a93e5e None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:38:39.477616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-9454bc33-ff5d-4c9e-9b5b-98c2a8a93e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:39.540294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:38:39.591400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 729/1489] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:38:39.906368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408dbde-41fe-40dc-b1b2-6ed4ec2dbf28 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:38:39.908218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3408dbde-41fe-40dc-b1b2-6ed4ec2dbf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:39.972627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:38:40.114715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408dbde-41fe-40dc-b1b2-6ed4ec2dbf28 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:38:40.348762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933de1e4-ff5b-49c0-85ad-e118372e574d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:38:40.349872 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933de1e4-ff5b-49c0-85ad-e118372e574d service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:40.350345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933de1e4-ff5b-49c0-85ad-e118372e574d service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:40.350816 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933de1e4-ff5b-49c0-85ad-e118372e574d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:40.354205 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-933de1e4-ff5b-49c0-85ad-e118372e574d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:40.496928 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:38:40.535985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933de1e4-ff5b-49c0-85ad-e118372e574d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:38:40.537828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933de1e4-ff5b-49c0-85ad-e118372e574d service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:38:40.561113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 761/1490] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:38:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 655 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:38:43.100300 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866ca6f-910d-4623-9fe3-d41ed8d41ab2 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:43.100838 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866ca6f-910d-4623-9fe3-d41ed8d41ab2 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:43.100838 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866ca6f-910d-4623-9fe3-d41ed8d41ab2 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:43.102224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 730/1491] 10.222.0.59 () {54 vars in 927 bytes} [Mon Aug 29 22:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:38:43.112492 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb09a8-a87b-4961-ba77-0c23ce844a20 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:43.113071 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb09a8-a87b-4961-ba77-0c23ce844a20 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:43.113417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb09a8-a87b-4961-ba77-0c23ce844a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:43.235427 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-4fcb09a8-a87b-4961-ba77-0c23ce844a20 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:38:43.237490 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-4fcb09a8-a87b-4961-ba77-0c23ce844a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:43.357038 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 762/1492] 10.222.0.59 () {58 vars in 1024 bytes} [Mon Aug 29 22:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:38:43.654597 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866ca6f-910d-4623-9fe3-d41ed8d41ab2 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:38:43.656261 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-5866ca6f-910d-4623-9fe3-d41ed8d41ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:43.775457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866ca6f-910d-4623-9fe3-d41ed8d41ab2 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:38:43.885280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4a35d-a5d0-4c24-b27e-a4e4269bcb3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:38:43.886774 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4a35d-a5d0-4c24-b27e-a4e4269bcb3d service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:38:43.887277 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4a35d-a5d0-4c24-b27e-a4e4269bcb3d service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:38:43.887805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4a35d-a5d0-4c24-b27e-a4e4269bcb3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:38:43.890913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-61a4a35d-a5d0-4c24-b27e-a4e4269bcb3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:38:43.906757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:38:44.048406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4a35d-a5d0-4c24-b27e-a4e4269bcb3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:38:44.049925 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4a35d-a5d0-4c24-b27e-a4e4269bcb3d service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:38:44.075102 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 731/1493] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:38:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:38:53.354476 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=82 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:38:54.072122 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=50 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:09.181825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47212e0-d11a-47ff-a3f5-94462cb34107 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:09.182420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47212e0-d11a-47ff-a3f5-94462cb34107 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:09.182420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47212e0-d11a-47ff-a3f5-94462cb34107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:09.258763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-a47212e0-d11a-47ff-a3f5-94462cb34107 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:09.259969 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a47212e0-d11a-47ff-a3f5-94462cb34107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:09.356328 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 763/1494] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:09.364466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e91742-aab4-4d8a-8abc-a8000bba7d88 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:09.364977 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e91742-aab4-4d8a-8abc-a8000bba7d88 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:09.365085 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e91742-aab4-4d8a-8abc-a8000bba7d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:09.443791 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-62e91742-aab4-4d8a-8abc-a8000bba7d88 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:09.445180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-62e91742-aab4-4d8a-8abc-a8000bba7d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:09.546217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 732/1495] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:09.551394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47212e0-d11a-47ff-a3f5-94462cb34107 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:09.552657 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a47212e0-d11a-47ff-a3f5-94462cb34107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:09.564425 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47212e0-d11a-47ff-a3f5-94462cb34107 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:09.627629 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:09.661743 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd976208-bbf2-4b11-9e63-d46ce6ce81cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:09.662345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd976208-bbf2-4b11-9e63-d46ce6ce81cd admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:09.662617 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd976208-bbf2-4b11-9e63-d46ce6ce81cd admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:09.662921 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd976208-bbf2-4b11-9e63-d46ce6ce81cd admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:09.663546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd976208-bbf2-4b11-9e63-d46ce6ce81cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:40:09.663836 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd976208-bbf2-4b11-9e63-d46ce6ce81cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:09.664878 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: WARNING py.warnings [None req-bd976208-bbf2-4b11-9e63-d46ce6ce81cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:40:09.664878 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: warnings.warn(msg) Aug 29 22:40:09.664878 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: Aug 29 22:40:09.665757 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd976208-bbf2-4b11-9e63-d46ce6ce81cd admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:09.677473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 764/1496] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:40:09.686544 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e91742-aab4-4d8a-8abc-a8000bba7d88 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:09.697295 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e91742-aab4-4d8a-8abc-a8000bba7d88 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:09.761323 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:09.795461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff3f01-3fac-4171-b958-81a297afe56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:09.796072 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff3f01-3fac-4171-b958-81a297afe56f admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:09.796372 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff3f01-3fac-4171-b958-81a297afe56f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:09.796634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff3f01-3fac-4171-b958-81a297afe56f admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:09.797290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff3f01-3fac-4171-b958-81a297afe56f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:09.798999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: WARNING py.warnings [None req-2eff3f01-3fac-4171-b958-81a297afe56f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:40:09.798999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: warnings.warn(msg) Aug 29 22:40:09.798999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: Aug 29 22:40:09.800226 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff3f01-3fac-4171-b958-81a297afe56f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:09.926374 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 733/1497] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:40:09 2022] POST /identity/v3/projects => generated 379 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:40:09.931290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd976208-bbf2-4b11-9e63-d46ce6ce81cd admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:09.932936 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-bd976208-bbf2-4b11-9e63-d46ce6ce81cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:09.968111 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd976208-bbf2-4b11-9e63-d46ce6ce81cd admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:10.069339 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cc269-e203-41a2-aca9-661f908532bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:10.070201 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cc269-e203-41a2-aca9-661f908532bf admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:10.070478 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cc269-e203-41a2-aca9-661f908532bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:10.070744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cc269-e203-41a2-aca9-661f908532bf admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:10.071379 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cc269-e203-41a2-aca9-661f908532bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:10.072413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: WARNING py.warnings [None req-160cc269-e203-41a2-aca9-661f908532bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:40:10.072413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: warnings.warn(msg) Aug 29 22:40:10.072413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: Aug 29 22:40:10.073618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cc269-e203-41a2-aca9-661f908532bf admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:10.184019 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 765/1498] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:40:09 2022] POST /identity/v3/users => generated 290 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:40:10.188399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff3f01-3fac-4171-b958-81a297afe56f admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:10.189921 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-2eff3f01-3fac-4171-b958-81a297afe56f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:10.253940 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff3f01-3fac-4171-b958-81a297afe56f admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:10.339990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:10.351797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cacc50-88cc-459a-9830-4cf9ebac72cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:10.352436 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cacc50-88cc-459a-9830-4cf9ebac72cf admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:10.352710 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cacc50-88cc-459a-9830-4cf9ebac72cf admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:10.352995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cacc50-88cc-459a-9830-4cf9ebac72cf admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:10.353628 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cacc50-88cc-459a-9830-4cf9ebac72cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:10.354655 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: WARNING py.warnings [None req-43cacc50-88cc-459a-9830-4cf9ebac72cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:40:10.354655 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: warnings.warn(msg) Aug 29 22:40:10.354655 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: Aug 29 22:40:10.355563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cacc50-88cc-459a-9830-4cf9ebac72cf admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:10.368422 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 734/1499] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:40:10.372919 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cc269-e203-41a2-aca9-661f908532bf admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:10.374459 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-160cc269-e203-41a2-aca9-661f908532bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:10.386122 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cc269-e203-41a2-aca9-661f908532bf admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:10.413714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:10.494825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0805414-937e-4dc9-a713-2df9831bedf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:10.496085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0805414-937e-4dc9-a713-2df9831bedf8 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:10.496638 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0805414-937e-4dc9-a713-2df9831bedf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:10.497140 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0805414-937e-4dc9-a713-2df9831bedf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:10.498332 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0805414-937e-4dc9-a713-2df9831bedf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:10.499386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: WARNING py.warnings [None req-c0805414-937e-4dc9-a713-2df9831bedf8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:40:10.499386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: warnings.warn(msg) Aug 29 22:40:10.499386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: Aug 29 22:40:10.500311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0805414-937e-4dc9-a713-2df9831bedf8 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:10.513751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 766/1500] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:40:10.518013 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cacc50-88cc-459a-9830-4cf9ebac72cf admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:10.519525 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-43cacc50-88cc-459a-9830-4cf9ebac72cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:10.555590 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cacc50-88cc-459a-9830-4cf9ebac72cf admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:10.657070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef8081-3d21-44e2-8ea4-b72816ae5495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:10.657809 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef8081-3d21-44e2-8ea4-b72816ae5495 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:10.658143 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef8081-3d21-44e2-8ea4-b72816ae5495 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:10.658420 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef8081-3d21-44e2-8ea4-b72816ae5495 admin admin] PATH_INFO: `/v3/projects/8a2561e80fce40ba9bba7f62cd7bf69e/users/bd397b6691e542e6a590fc84de50256f/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:10.682049 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef8081-3d21-44e2-8ea4-b72816ae5495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a2561e80fce40ba9bba7f62cd7bf69e, user_id=bd397b6691e542e6a590fc84de50256f, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:10.683081 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: WARNING py.warnings [None req-d1ef8081-3d21-44e2-8ea4-b72816ae5495 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:40:10.683081 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: warnings.warn(msg) Aug 29 22:40:10.683081 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: Aug 29 22:40:10.685141 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef8081-3d21-44e2-8ea4-b72816ae5495 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:10.763915 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 735/1501] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:40:10 2022] PUT /identity/v3/projects/8a2561e80fce40ba9bba7f62cd7bf69e/users/bd397b6691e542e6a590fc84de50256f/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:40:10.776453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6737967-761e-4d02-a272-613122ddbdae None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:10.776727 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6737967-761e-4d02-a272-613122ddbdae None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:10.776998 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6737967-761e-4d02-a272-613122ddbdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:10.836948 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-a6737967-761e-4d02-a272-613122ddbdae None None] MFA Rules not processed for user `bd397b6691e542e6a590fc84de50256f`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:10.838178 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a6737967-761e-4d02-a272-613122ddbdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:10.918712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 767/1502] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:40:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:10.933780 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bb173-9ab1-4601-8dda-48f73f81b096 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:10.934067 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bb173-9ab1-4601-8dda-48f73f81b096 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:10.934397 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bb173-9ab1-4601-8dda-48f73f81b096 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:10.935668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 736/1503] 10.222.0.59 () {54 vars in 964 bytes} [Mon Aug 29 22:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:40:10.940629 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0a6a9-4fc4-4f1f-b978-228f334c0ab3 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:10.940910 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0a6a9-4fc4-4f1f-b978-228f334c0ab3 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:10.941182 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0a6a9-4fc4-4f1f-b978-228f334c0ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:10.973539 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:11.022024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-a4f0a6a9-4fc4-4f1f-b978-228f334c0ab3 None None] MFA Rules not processed for user `92af56c889e640e7815c1641c05a9af3`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:11.023278 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a4f0a6a9-4fc4-4f1f-b978-228f334c0ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:11.103433 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 768/1504] 10.222.0.59 () {58 vars in 1061 bytes} [Mon Aug 29 22:40:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:11.111608 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bb173-9ab1-4601-8dda-48f73f81b096 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:11.113176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-5d5bb173-9ab1-4601-8dda-48f73f81b096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:11.173167 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:11.231341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bb173-9ab1-4601-8dda-48f73f81b096 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:11.309172 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b50a7-59e8-4734-85ca-57f89da93820 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:11.309717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b50a7-59e8-4734-85ca-57f89da93820 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:11.309995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b50a7-59e8-4734-85ca-57f89da93820 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:11.310261 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b50a7-59e8-4734-85ca-57f89da93820 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:11.311870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-377b50a7-59e8-4734-85ca-57f89da93820 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:11.420704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b50a7-59e8-4734-85ca-57f89da93820 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:11.422879 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b50a7-59e8-4734-85ca-57f89da93820 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:11.425933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:11.440978 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 737/1505] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:40:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:40:11.491550 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad2772-c995-448d-ad0c-e634744eb330 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:11.492594 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad2772-c995-448d-ad0c-e634744eb330 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:11.492594 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad2772-c995-448d-ad0c-e634744eb330 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:11.493967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 769/1506] 10.222.0.59 () {52 vars in 830 bytes} [Mon Aug 29 22:40:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:40:11.498359 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b50a7-59e8-4734-85ca-57f89da93820 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:11.499613 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-377b50a7-59e8-4734-85ca-57f89da93820 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:11.614747 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b50a7-59e8-4734-85ca-57f89da93820 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:11.715356 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c6380-3c69-44a9-9e3d-34d93c5c24f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:11.715981 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c6380-3c69-44a9-9e3d-34d93c5c24f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:11.716285 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c6380-3c69-44a9-9e3d-34d93c5c24f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:11.716554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c6380-3c69-44a9-9e3d-34d93c5c24f0 admin admin] PATH_INFO: `/v3/users/bd397b6691e542e6a590fc84de50256f` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:11.730222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c6380-3c69-44a9-9e3d-34d93c5c24f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd397b6691e542e6a590fc84de50256f)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:11.731235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: WARNING py.warnings [None req-284c6380-3c69-44a9-9e3d-34d93c5c24f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:40:11.731235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: warnings.warn(msg) Aug 29 22:40:11.731235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: Aug 29 22:40:11.732479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c6380-3c69-44a9-9e3d-34d93c5c24f0 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:11.743165 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:11.944442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-284c6380-3c69-44a9-9e3d-34d93c5c24f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd397b6691e542e6a590fc84de50256f'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:40:11.959023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-284c6380-3c69-44a9-9e3d-34d93c5c24f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd397b6691e542e6a590fc84de50256f'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:40:11.960114 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-284c6380-3c69-44a9-9e3d-34d93c5c24f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd397b6691e542e6a590fc84de50256f'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:40:11.985094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-284c6380-3c69-44a9-9e3d-34d93c5c24f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd397b6691e542e6a590fc84de50256f'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:40:11.985404 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-284c6380-3c69-44a9-9e3d-34d93c5c24f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd397b6691e542e6a590fc84de50256f'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:40:12.053401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 738/1507] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:40:11 2022] DELETE /identity/v3/users/bd397b6691e542e6a590fc84de50256f => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:40:12.057951 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af932b4a-74ed-4422-8558-15ca9a595140 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:12.058238 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af932b4a-74ed-4422-8558-15ca9a595140 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:12.058540 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af932b4a-74ed-4422-8558-15ca9a595140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:12.116499 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-af932b4a-74ed-4422-8558-15ca9a595140 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:40:12.117841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-af932b4a-74ed-4422-8558-15ca9a595140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:12.191310 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:12.203568 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 770/1508] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:40:12.214085 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c6380-3c69-44a9-9e3d-34d93c5c24f0 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:12.215297 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-284c6380-3c69-44a9-9e3d-34d93c5c24f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:12.318143 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c6380-3c69-44a9-9e3d-34d93c5c24f0 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:12.373945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:12.394422 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ecd1d-fbdd-4f8e-9d4d-ab4a9eb1d33c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:12.395066 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ecd1d-fbdd-4f8e-9d4d-ab4a9eb1d33c service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:12.395343 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ecd1d-fbdd-4f8e-9d4d-ab4a9eb1d33c service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:12.395612 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ecd1d-fbdd-4f8e-9d4d-ab4a9eb1d33c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:12.397987 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-1e5ecd1d-fbdd-4f8e-9d4d-ab4a9eb1d33c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:12.513814 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ecd1d-fbdd-4f8e-9d4d-ab4a9eb1d33c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:12.515154 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ecd1d-fbdd-4f8e-9d4d-ab4a9eb1d33c service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:12.534377 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 739/1509] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:40:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:40:12.988619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af932b4a-74ed-4422-8558-15ca9a595140 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:40:12.990131 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-af932b4a-74ed-4422-8558-15ca9a595140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:40:13.002320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af932b4a-74ed-4422-8558-15ca9a595140 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:40:13.101955 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43069f7b-c34f-4d1a-af41-16df0270d7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:40:13.102585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43069f7b-c34f-4d1a-af41-16df0270d7ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:40:13.102860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43069f7b-c34f-4d1a-af41-16df0270d7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:40:13.103128 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43069f7b-c34f-4d1a-af41-16df0270d7ab admin admin] PATH_INFO: `/v3/projects/8a2561e80fce40ba9bba7f62cd7bf69e` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:40:13.113180 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43069f7b-c34f-4d1a-af41-16df0270d7ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a2561e80fce40ba9bba7f62cd7bf69e)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:40:13.114199 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: WARNING py.warnings [None req-43069f7b-c34f-4d1a-af41-16df0270d7ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:40:13.114199 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: warnings.warn(msg) Aug 29 22:40:13.114199 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: Aug 29 22:40:13.115392 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43069f7b-c34f-4d1a-af41-16df0270d7ab admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:40:13.231534 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-43069f7b-c34f-4d1a-af41-16df0270d7ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a2561e80fce40ba9bba7f62cd7bf69e', 'request_id': 'req-43069f7b-c34f-4d1a-af41-16df0270d7ab'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:40:13.244376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-43069f7b-c34f-4d1a-af41-16df0270d7ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a2561e80fce40ba9bba7f62cd7bf69e', 'request_id': 'req-43069f7b-c34f-4d1a-af41-16df0270d7ab'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:40:13.250386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-43069f7b-c34f-4d1a-af41-16df0270d7ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a2561e80fce40ba9bba7f62cd7bf69e', 'request_id': 'req-43069f7b-c34f-4d1a-af41-16df0270d7ab'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:40:13.342140 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-43069f7b-c34f-4d1a-af41-16df0270d7ab admin admin] The token cache is being invalidate because project 8a2561e80fce40ba9bba7f62cd7bf69e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:40:13.342359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-43069f7b-c34f-4d1a-af41-16df0270d7ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:40:13.342738 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-43069f7b-c34f-4d1a-af41-16df0270d7ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:40:13.344510 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 771/1510] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:40:12 2022] DELETE /identity/v3/projects/8a2561e80fce40ba9bba7f62cd7bf69e => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:40:22.532024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=62 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:40:23.340092 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:44.814005 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866307fa-30b9-4c02-a6f5-ab2604dfeb6b None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:44.814648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866307fa-30b9-4c02-a6f5-ab2604dfeb6b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:44.814648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866307fa-30b9-4c02-a6f5-ab2604dfeb6b None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:44.815894 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 740/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:44.821652 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7ae09-2b00-419e-91b9-7011bbc72250 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:44.822146 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7ae09-2b00-419e-91b9-7011bbc72250 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:44.822265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7ae09-2b00-419e-91b9-7011bbc72250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:44.912448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-47d7ae09-2b00-419e-91b9-7011bbc72250 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:44.913932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-47d7ae09-2b00-419e-91b9-7011bbc72250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:45.033371 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 772/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:45.191083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866307fa-30b9-4c02-a6f5-ab2604dfeb6b None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:45.193735 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-866307fa-30b9-4c02-a6f5-ab2604dfeb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:45.338476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866307fa-30b9-4c02-a6f5-ab2604dfeb6b None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:45.344513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbe4a7-970b-4f2c-a7ae-ef57f9c54b55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:45.347259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbe4a7-970b-4f2c-a7ae-ef57f9c54b55 service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:45.347259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbe4a7-970b-4f2c-a7ae-ef57f9c54b55 service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:45.347259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbe4a7-970b-4f2c-a7ae-ef57f9c54b55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:45.356867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bbe4a7-970b-4f2c-a7ae-ef57f9c54b55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:45.359827 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bbe4a7-970b-4f2c-a7ae-ef57f9c54b55 service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:45.374899 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 741/1513] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:46.258338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4421e-12ad-4f0c-b051-ce4f7f454f43 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:46.258943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4421e-12ad-4f0c-b051-ce4f7f454f43 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:46.258943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4421e-12ad-4f0c-b051-ce4f7f454f43 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:46.260172 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 773/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:41:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:46.265628 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74de542-e45b-4a6f-9a6e-b53951d4d0f0 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:46.266137 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74de542-e45b-4a6f-9a6e-b53951d4d0f0 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:46.266239 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74de542-e45b-4a6f-9a6e-b53951d4d0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:46.287795 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-f74de542-e45b-4a6f-9a6e-b53951d4d0f0 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:46.289086 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f74de542-e45b-4a6f-9a6e-b53951d4d0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:46.297701 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 742/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:41:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:46.455821 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4421e-12ad-4f0c-b051-ce4f7f454f43 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:46.472463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4421e-12ad-4f0c-b051-ce4f7f454f43 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:46.478833 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53390d21-38ba-4156-bb8d-3ce90dd25c5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:46.479444 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53390d21-38ba-4156-bb8d-3ce90dd25c5f service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:46.479712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53390d21-38ba-4156-bb8d-3ce90dd25c5f service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:46.479979 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53390d21-38ba-4156-bb8d-3ce90dd25c5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:46.488934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53390d21-38ba-4156-bb8d-3ce90dd25c5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:46.491473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53390d21-38ba-4156-bb8d-3ce90dd25c5f service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:46.501545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 774/1516] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:47.560508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f787b1-4103-4eeb-9efd-9142eb8ef902 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:47.560508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f787b1-4103-4eeb-9efd-9142eb8ef902 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:47.560508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f787b1-4103-4eeb-9efd-9142eb8ef902 None None] PATH_INFO: `/v3` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:47.560508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 743/1517] 10.222.0.40 () {54 vars in 925 bytes} [Mon Aug 29 22:41:47 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:41:47.597933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60fd12-d4a7-460d-a9b5-77382e7930e6 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:47.598724 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60fd12-d4a7-460d-a9b5-77382e7930e6 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:47.598724 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60fd12-d4a7-460d-a9b5-77382e7930e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:47.667668 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-4f60fd12-d4a7-460d-a9b5-77382e7930e6 None None] MFA Rules not processed for user `7215ee41033742d79671f4f42b81cd06`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:47.669643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-4f60fd12-d4a7-460d-a9b5-77382e7930e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:47.696645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:41:47.767863 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 775/1518] 10.222.0.40 () {58 vars in 1015 bytes} [Mon Aug 29 22:41:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:41:52.016751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca634a-8cf3-4cbf-b2b6-7fbb44fa42a9 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:52.017276 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca634a-8cf3-4cbf-b2b6-7fbb44fa42a9 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:52.017276 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca634a-8cf3-4cbf-b2b6-7fbb44fa42a9 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:52.018645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 744/1519] 10.222.0.59 () {54 vars in 961 bytes} [Mon Aug 29 22:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:52.024028 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f48b97-94b5-4718-85c5-8f2e9259ca23 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:52.024486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f48b97-94b5-4718-85c5-8f2e9259ca23 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:52.024602 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f48b97-94b5-4718-85c5-8f2e9259ca23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:52.044253 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-c4f48b97-94b5-4718-85c5-8f2e9259ca23 None None] MFA Rules not processed for user `7215ee41033742d79671f4f42b81cd06`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:52.045521 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c4f48b97-94b5-4718-85c5-8f2e9259ca23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:52.054620 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 776/1520] 10.222.0.59 () {58 vars in 1058 bytes} [Mon Aug 29 22:41:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:52.063134 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca634a-8cf3-4cbf-b2b6-7fbb44fa42a9 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:52.078081 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca634a-8cf3-4cbf-b2b6-7fbb44fa42a9 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:52.082964 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c9e7f-be1c-421c-b346-354363c19a99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:52.083557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c9e7f-be1c-421c-b346-354363c19a99 service placement] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:52.083833 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c9e7f-be1c-421c-b346-354363c19a99 service placement] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:52.084149 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c9e7f-be1c-421c-b346-354363c19a99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:52.093203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c9e7f-be1c-421c-b346-354363c19a99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:52.094776 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c9e7f-be1c-421c-b346-354363c19a99 service placement] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:52.104951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 745/1521] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:57.712569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb04db-3d84-45dd-8a74-889beda9dbf6 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:57.713274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb04db-3d84-45dd-8a74-889beda9dbf6 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:57.713274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb04db-3d84-45dd-8a74-889beda9dbf6 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:57.714441 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 777/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:57.719905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10786c-a4ba-449f-9d8b-9e676cbf3588 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:57.720435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10786c-a4ba-449f-9d8b-9e676cbf3588 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:57.720540 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10786c-a4ba-449f-9d8b-9e676cbf3588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:57.742407 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-6f10786c-a4ba-449f-9d8b-9e676cbf3588 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:57.743627 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6f10786c-a4ba-449f-9d8b-9e676cbf3588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:57.751826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 746/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:57.910860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb04db-3d84-45dd-8a74-889beda9dbf6 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:57.927332 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb04db-3d84-45dd-8a74-889beda9dbf6 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:57.935105 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640bac4f-2ccb-4a43-9aed-1e57e7ed2637 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:57.935723 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bac4f-2ccb-4a43-9aed-1e57e7ed2637 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:57.936017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bac4f-2ccb-4a43-9aed-1e57e7ed2637 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:57.936325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bac4f-2ccb-4a43-9aed-1e57e7ed2637 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:57.945993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640bac4f-2ccb-4a43-9aed-1e57e7ed2637 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:57.947643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640bac4f-2ccb-4a43-9aed-1e57e7ed2637 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:57.958232 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 778/1524] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:41:58.819499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f78a63-626e-45b5-a29b-d827cc19753c None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:58.820176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f78a63-626e-45b5-a29b-d827cc19753c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:58.820176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f78a63-626e-45b5-a29b-d827cc19753c None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:58.821501 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 747/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 22:41:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:41:58.826687 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10354b3-cabd-45a0-8725-3e33e0f7a4a3 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:58.827162 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10354b3-cabd-45a0-8725-3e33e0f7a4a3 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:58.827368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10354b3-cabd-45a0-8725-3e33e0f7a4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:58.848188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-a10354b3-cabd-45a0-8725-3e33e0f7a4a3 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:41:58.849463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a10354b3-cabd-45a0-8725-3e33e0f7a4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:41:58.858635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 779/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 22:41:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:41:59.013391 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f78a63-626e-45b5-a29b-d827cc19753c None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:41:59.029013 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f78a63-626e-45b5-a29b-d827cc19753c None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:41:59.034626 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1b3c2-74d4-469f-87dd-ac3ee2e9f9b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:41:59.035258 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1b3c2-74d4-469f-87dd-ac3ee2e9f9b0 service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:41:59.035532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1b3c2-74d4-469f-87dd-ac3ee2e9f9b0 service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:41:59.035802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1b3c2-74d4-469f-87dd-ac3ee2e9f9b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:41:59.045571 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1b3c2-74d4-469f-87dd-ac3ee2e9f9b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:41:59.047175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1b3c2-74d4-469f-87dd-ac3ee2e9f9b0 service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:41:59.058358 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 748/1527] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:41:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:42:02.839718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c620171-ea1d-4d38-80c3-fed6d440eaf4 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:02.840319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c620171-ea1d-4d38-80c3-fed6d440eaf4 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:02.840319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c620171-ea1d-4d38-80c3-fed6d440eaf4 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:02.841698 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 780/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:42:02.847493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311eaa5c-74fc-40a5-accf-1cb0b3a87973 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:02.847971 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311eaa5c-74fc-40a5-accf-1cb0b3a87973 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:02.848255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311eaa5c-74fc-40a5-accf-1cb0b3a87973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:02.869704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-311eaa5c-74fc-40a5-accf-1cb0b3a87973 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:02.870933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-311eaa5c-74fc-40a5-accf-1cb0b3a87973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:02.879587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 749/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:42:03.189822 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c620171-ea1d-4d38-80c3-fed6d440eaf4 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:42:03.191537 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-2c620171-ea1d-4d38-80c3-fed6d440eaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:03.277298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c620171-ea1d-4d38-80c3-fed6d440eaf4 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:42:03.281058 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544fd24-940d-4b21-b0a8-9f8d2c658971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:42:03.283785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544fd24-940d-4b21-b0a8-9f8d2c658971 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:03.283785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544fd24-940d-4b21-b0a8-9f8d2c658971 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:03.283785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544fd24-940d-4b21-b0a8-9f8d2c658971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:03.292619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544fd24-940d-4b21-b0a8-9f8d2c658971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:42:03.294060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544fd24-940d-4b21-b0a8-9f8d2c658971 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:42:03.303710 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 781/1530] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:42:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:42:05.276041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b00956-7295-46b3-8a8c-c836fd4691c3 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:05.276661 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b00956-7295-46b3-8a8c-c836fd4691c3 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:05.276661 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b00956-7295-46b3-8a8c-c836fd4691c3 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:05.277874 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 750/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:42:05.283409 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f26191-d8d0-4252-8419-0cd6648b5cf5 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:05.284092 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f26191-d8d0-4252-8419-0cd6648b5cf5 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:05.284092 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f26191-d8d0-4252-8419-0cd6648b5cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:05.304280 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-b3f26191-d8d0-4252-8419-0cd6648b5cf5 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:05.305484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b3f26191-d8d0-4252-8419-0cd6648b5cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:05.313314 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 782/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:42:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:42:05.608489 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b00956-7295-46b3-8a8c-c836fd4691c3 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:42:05.620711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b00956-7295-46b3-8a8c-c836fd4691c3 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:42:05.624330 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc240b-9b6c-4db0-a408-d2ee35785ead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:42:05.624933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc240b-9b6c-4db0-a408-d2ee35785ead service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:05.625209 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc240b-9b6c-4db0-a408-d2ee35785ead service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:05.625479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc240b-9b6c-4db0-a408-d2ee35785ead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:05.635750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc240b-9b6c-4db0-a408-d2ee35785ead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:42:05.637197 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc240b-9b6c-4db0-a408-d2ee35785ead service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:42:05.647927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 751/1533] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:42:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:42:09.398615 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82b975-94fd-4da7-85c1-b0198861ecd0 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:09.399191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82b975-94fd-4da7-85c1-b0198861ecd0 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:09.399191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82b975-94fd-4da7-85c1-b0198861ecd0 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:09.400643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 783/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:42:09.406204 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab64bd-413f-4964-96a8-d4156e0ccd49 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:09.406710 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab64bd-413f-4964-96a8-d4156e0ccd49 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:09.406821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab64bd-413f-4964-96a8-d4156e0ccd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:09.428953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-ebab64bd-413f-4964-96a8-d4156e0ccd49 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:09.430170 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ebab64bd-413f-4964-96a8-d4156e0ccd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:09.438388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 752/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:42:10.004422 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82b975-94fd-4da7-85c1-b0198861ecd0 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:42:10.006378 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-7f82b975-94fd-4da7-85c1-b0198861ecd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:10.074945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:10.089452 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82b975-94fd-4da7-85c1-b0198861ecd0 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:42:10.093178 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf7a16f-4727-46f3-9195-8f9a7c97dae4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:42:10.093772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7a16f-4727-46f3-9195-8f9a7c97dae4 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:10.094052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7a16f-4727-46f3-9195-8f9a7c97dae4 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:10.094317 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7a16f-4727-46f3-9195-8f9a7c97dae4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:10.104838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf7a16f-4727-46f3-9195-8f9a7c97dae4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:42:10.106264 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf7a16f-4727-46f3-9195-8f9a7c97dae4 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:42:10.117151 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 784/1536] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:42:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:42:12.083958 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d53b2-1596-49d9-8f6b-8083e3de2c91 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:12.084529 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d53b2-1596-49d9-8f6b-8083e3de2c91 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:12.084529 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d53b2-1596-49d9-8f6b-8083e3de2c91 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:12.085887 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 753/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 22:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:42:12.091207 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de126f85-07f2-4b9b-ab23-ee0e98e7279e None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:12.091617 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de126f85-07f2-4b9b-ab23-ee0e98e7279e None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:12.091740 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de126f85-07f2-4b9b-ab23-ee0e98e7279e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:12.112245 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-de126f85-07f2-4b9b-ab23-ee0e98e7279e None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:42:12.113673 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-de126f85-07f2-4b9b-ab23-ee0e98e7279e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:12.122987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 785/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 22:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:42:12.693991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d53b2-1596-49d9-8f6b-8083e3de2c91 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:42:12.695705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a06d53b2-1596-49d9-8f6b-8083e3de2c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:42:12.703377 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=84 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:12.710445 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d53b2-1596-49d9-8f6b-8083e3de2c91 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:42:12.714502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe86c2b-4688-439a-b316-69c72dfe8745 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:42:12.725383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe86c2b-4688-439a-b316-69c72dfe8745 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:42:12.725383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe86c2b-4688-439a-b316-69c72dfe8745 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:42:12.725383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe86c2b-4688-439a-b316-69c72dfe8745 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:42:12.725383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe86c2b-4688-439a-b316-69c72dfe8745 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:42:12.726578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe86c2b-4688-439a-b316-69c72dfe8745 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:42:12.736955 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 754/1539] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:42:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:42:22.107454 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=11 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:42:22.737990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=5 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:39.778707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfbffd-a67b-46de-8fb2-6c63f9883a30 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:39.779323 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfbffd-a67b-46de-8fb2-6c63f9883a30 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:39.779323 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfbffd-a67b-46de-8fb2-6c63f9883a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:39.803035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-f2dfbffd-a67b-46de-8fb2-6c63f9883a30 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:39.808506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f2dfbffd-a67b-46de-8fb2-6c63f9883a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:39.824339 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 786/1540] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:39.833402 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be2cce-4614-4f8e-984f-3142cfe47eae None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:39.834286 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be2cce-4614-4f8e-984f-3142cfe47eae None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:39.834286 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be2cce-4614-4f8e-984f-3142cfe47eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:39.869361 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-89be2cce-4614-4f8e-984f-3142cfe47eae None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:39.870616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-89be2cce-4614-4f8e-984f-3142cfe47eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:39.881670 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 755/1541] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:39.887985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfbffd-a67b-46de-8fb2-6c63f9883a30 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:39.898221 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfbffd-a67b-46de-8fb2-6c63f9883a30 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:39.908844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9a562-b0f2-4589-910b-1adc97fc5dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:39.909470 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9a562-b0f2-4589-910b-1adc97fc5dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:39.909748 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9a562-b0f2-4589-910b-1adc97fc5dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:39.910015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9a562-b0f2-4589-910b-1adc97fc5dd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:39.910674 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9a562-b0f2-4589-910b-1adc97fc5dd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:43:39.910975 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9a562-b0f2-4589-910b-1adc97fc5dd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:39.912632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9a562-b0f2-4589-910b-1adc97fc5dd9 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:39.930133 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 787/1542] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:43:39.938272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be2cce-4614-4f8e-984f-3142cfe47eae None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:39.953953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be2cce-4614-4f8e-984f-3142cfe47eae None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:39.970623 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e860adf-ae66-417c-85f4-6b5ecab9c6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:39.971314 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e860adf-ae66-417c-85f4-6b5ecab9c6f1 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:39.971591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e860adf-ae66-417c-85f4-6b5ecab9c6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:39.971859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e860adf-ae66-417c-85f4-6b5ecab9c6f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:39.972532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e860adf-ae66-417c-85f4-6b5ecab9c6f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:39.974470 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e860adf-ae66-417c-85f4-6b5ecab9c6f1 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:40.106293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 756/1543] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:43:39 2022] POST /identity/v3/projects => generated 389 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:40.110747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9a562-b0f2-4589-910b-1adc97fc5dd9 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:40.124449 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9a562-b0f2-4589-910b-1adc97fc5dd9 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:40.130660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de912cf8-00ae-4949-b21e-b278988e0188 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.131155 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de912cf8-00ae-4949-b21e-b278988e0188 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.131626 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de912cf8-00ae-4949-b21e-b278988e0188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.176046 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-de912cf8-00ae-4949-b21e-b278988e0188 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:40.180307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-de912cf8-00ae-4949-b21e-b278988e0188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:40.267666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67b5f3-ce23-4515-ae79-07485668d550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:40.268865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67b5f3-ce23-4515-ae79-07485668d550 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.268865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67b5f3-ce23-4515-ae79-07485668d550 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.269110 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67b5f3-ce23-4515-ae79-07485668d550 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.269704 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67b5f3-ce23-4515-ae79-07485668d550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:40.271706 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67b5f3-ce23-4515-ae79-07485668d550 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:40.330123 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 757/1544] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:40.331574 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f88d41-e8f4-4dcd-adaf-b09fd5eb3a97 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.331856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f88d41-e8f4-4dcd-adaf-b09fd5eb3a97 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.332130 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f88d41-e8f4-4dcd-adaf-b09fd5eb3a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.373449 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-c8f88d41-e8f4-4dcd-adaf-b09fd5eb3a97 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:40.374694 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-c8f88d41-e8f4-4dcd-adaf-b09fd5eb3a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:40.376874 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 788/1545] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:43:40 2022] POST /identity/v3/users => generated 295 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:40.378434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4872e-6592-4a55-957e-c48c11cce7d7 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.378656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4872e-6592-4a55-957e-c48c11cce7d7 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.379459 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4872e-6592-4a55-957e-c48c11cce7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.397978 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 758/1546] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:40.398869 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f88d41-e8f4-4dcd-adaf-b09fd5eb3a97 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:40.420494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f88d41-e8f4-4dcd-adaf-b09fd5eb3a97 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:40.421249 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-64d4872e-6592-4a55-957e-c48c11cce7d7 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:40.423204 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-64d4872e-6592-4a55-957e-c48c11cce7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:40.435265 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a390a-18cb-4556-960b-5a1360689ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:40.435899 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a390a-18cb-4556-960b-5a1360689ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.436193 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a390a-18cb-4556-960b-5a1360689ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.436460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a390a-18cb-4556-960b-5a1360689ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.437132 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a390a-18cb-4556-960b-5a1360689ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:40.438752 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a390a-18cb-4556-960b-5a1360689ce3 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:40.441399 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 789/1547] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:40.443613 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed3d68-77a7-4e56-894a-dd6d7b55aa12 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.443873 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed3d68-77a7-4e56-894a-dd6d7b55aa12 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.444186 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed3d68-77a7-4e56-894a-dd6d7b55aa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.465533 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 759/1548] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:40.471741 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312be0e-27e4-4f01-86af-13a655be0c36 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.472046 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312be0e-27e4-4f01-86af-13a655be0c36 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.472305 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312be0e-27e4-4f01-86af-13a655be0c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.485142 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-65ed3d68-77a7-4e56-894a-dd6d7b55aa12 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:40.486398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-65ed3d68-77a7-4e56-894a-dd6d7b55aa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:40.512844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 790/1549] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:40.517710 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed3d68-77a7-4e56-894a-dd6d7b55aa12 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:40.529999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-9312be0e-27e4-4f01-86af-13a655be0c36 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:40.531533 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-9312be0e-27e4-4f01-86af-13a655be0c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:40.533505 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed3d68-77a7-4e56-894a-dd6d7b55aa12 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:40.539012 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab9bbb-08a6-41b9-b2e0-324f8a426e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:40.540264 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab9bbb-08a6-41b9-b2e0-324f8a426e49 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.540551 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab9bbb-08a6-41b9-b2e0-324f8a426e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.540880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab9bbb-08a6-41b9-b2e0-324f8a426e49 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.542017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab9bbb-08a6-41b9-b2e0-324f8a426e49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:43:40.542136 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 760/1550] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:40.542637 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312be0e-27e4-4f01-86af-13a655be0c36 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:40.544297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab9bbb-08a6-41b9-b2e0-324f8a426e49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:40.547447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab9bbb-08a6-41b9-b2e0-324f8a426e49 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:40.553560 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312be0e-27e4-4f01-86af-13a655be0c36 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:40.563242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d95d3-9595-4131-8271-bafbc7f1d977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:40.563892 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d95d3-9595-4131-8271-bafbc7f1d977 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.564176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d95d3-9595-4131-8271-bafbc7f1d977 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.564454 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d95d3-9595-4131-8271-bafbc7f1d977 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.565094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d95d3-9595-4131-8271-bafbc7f1d977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:40.572342 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 791/1551] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:43:40.572342 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab9bbb-08a6-41b9-b2e0-324f8a426e49 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:40.572711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d95d3-9595-4131-8271-bafbc7f1d977 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:40.580561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 761/1552] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:40.581888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7378cef-6ab9-49bc-b40e-910ec72c949e None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.582290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7378cef-6ab9-49bc-b40e-910ec72c949e None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.582558 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7378cef-6ab9-49bc-b40e-910ec72c949e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.588336 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab9bbb-08a6-41b9-b2e0-324f8a426e49 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:40.595691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88458596-3e36-4335-b63e-1630a14cbfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:40.596337 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88458596-3e36-4335-b63e-1630a14cbfdc admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.596667 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88458596-3e36-4335-b63e-1630a14cbfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.596867 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88458596-3e36-4335-b63e-1630a14cbfdc admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.597515 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88458596-3e36-4335-b63e-1630a14cbfdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:43:40.597810 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88458596-3e36-4335-b63e-1630a14cbfdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:40.599416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88458596-3e36-4335-b63e-1630a14cbfdc admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:40.609408 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-b7378cef-6ab9-49bc-b40e-910ec72c949e None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:40.610380 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b7378cef-6ab9-49bc-b40e-910ec72c949e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:40.616200 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 792/1553] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:43:40.616200 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88dd8c-d64f-42cc-a594-28cc205b3273 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.616200 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88dd8c-d64f-42cc-a594-28cc205b3273 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.616200 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88dd8c-d64f-42cc-a594-28cc205b3273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.629982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 762/1554] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:40.631538 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4a874-0f9d-44b4-8e83-3123aae0dff8 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.631855 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4a874-0f9d-44b4-8e83-3123aae0dff8 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.632206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4a874-0f9d-44b4-8e83-3123aae0dff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.641152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-7a88dd8c-d64f-42cc-a594-28cc205b3273 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:40.642364 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-7a88dd8c-d64f-42cc-a594-28cc205b3273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:40.653188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 793/1555] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:40.654659 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc97035-c497-4234-ad5f-f3ab7b5fa2fc None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.654785 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-cea4a874-0f9d-44b4-8e83-3123aae0dff8 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:40.654999 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc97035-c497-4234-ad5f-f3ab7b5fa2fc None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.655208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc97035-c497-4234-ad5f-f3ab7b5fa2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.655917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-cea4a874-0f9d-44b4-8e83-3123aae0dff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:40.664595 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 763/1556] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:40.665269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4a874-0f9d-44b4-8e83-3123aae0dff8 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:40.675475 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4a874-0f9d-44b4-8e83-3123aae0dff8 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:40.679677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-cfc97035-c497-4234-ad5f-f3ab7b5fa2fc None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:40.680277 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181917f-12d1-42f9-b0f0-41cefce0db5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:40.680902 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181917f-12d1-42f9-b0f0-41cefce0db5f admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.681172 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181917f-12d1-42f9-b0f0-41cefce0db5f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.681404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-cfc97035-c497-4234-ad5f-f3ab7b5fa2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:40.681530 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181917f-12d1-42f9-b0f0-41cefce0db5f admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.682079 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181917f-12d1-42f9-b0f0-41cefce0db5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:40.683969 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181917f-12d1-42f9-b0f0-41cefce0db5f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:40.690161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 794/1557] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:40.690438 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc97035-c497-4234-ad5f-f3ab7b5fa2fc None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:40.702624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc97035-c497-4234-ad5f-f3ab7b5fa2fc None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:40.707615 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5dd93-6a23-41ee-be62-618f929623dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:40.708432 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5dd93-6a23-41ee-be62-618f929623dd admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.708712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5dd93-6a23-41ee-be62-618f929623dd admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.708979 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5dd93-6a23-41ee-be62-618f929623dd admin admin] PATH_INFO: `/v3/projects/d45f8dd557e14a6483ebe7e39d902641/users/42a287a54c4040ac828ee221a3f2e8c7/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.726074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5dd93-6a23-41ee-be62-618f929623dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d45f8dd557e14a6483ebe7e39d902641, user_id=42a287a54c4040ac828ee221a3f2e8c7, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:40.727142 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: WARNING py.warnings [None req-22d5dd93-6a23-41ee-be62-618f929623dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:43:40.727142 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: warnings.warn(msg) Aug 29 22:43:40.727142 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: Aug 29 22:43:40.729614 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d5dd93-6a23-41ee-be62-618f929623dd admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:40.791682 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 764/1558] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:43:40 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:40.792658 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181917f-12d1-42f9-b0f0-41cefce0db5f admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:40.818985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 795/1559] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:43:40 2022] PUT /identity/v3/projects/d45f8dd557e14a6483ebe7e39d902641/users/42a287a54c4040ac828ee221a3f2e8c7/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:40.820484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8837a-f67c-48a8-9d2a-65189d5f2a57 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.824790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8837a-f67c-48a8-9d2a-65189d5f2a57 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.824790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8837a-f67c-48a8-9d2a-65189d5f2a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.843683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-43a8837a-f67c-48a8-9d2a-65189d5f2a57 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:40.845014 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-43a8837a-f67c-48a8-9d2a-65189d5f2a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:40.849404 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:40.860157 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181917f-12d1-42f9-b0f0-41cefce0db5f admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:40.892609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:40.905610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 796/1560] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:40.906903 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5194db6-9ea9-410b-ac61-4558f14cefd9 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.907335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5194db6-9ea9-410b-ac61-4558f14cefd9 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.907459 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5194db6-9ea9-410b-ac61-4558f14cefd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.929895 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-f5194db6-9ea9-410b-ac61-4558f14cefd9 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:40.931528 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a3d91-6d4b-41b2-9585-8c8e856717ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:40.932101 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f5194db6-9ea9-410b-ac61-4558f14cefd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:40.937895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a3d91-6d4b-41b2-9585-8c8e856717ba admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.937895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a3d91-6d4b-41b2-9585-8c8e856717ba admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.937895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a3d91-6d4b-41b2-9585-8c8e856717ba admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.937895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a3d91-6d4b-41b2-9585-8c8e856717ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:40.937895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a3d91-6d4b-41b2-9585-8c8e856717ba admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:40.941226 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 797/1561] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:40.941799 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81c88c-cd00-4f72-80f1-78971a010ff1 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.942056 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81c88c-cd00-4f72-80f1-78971a010ff1 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.942317 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81c88c-cd00-4f72-80f1-78971a010ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.963190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-9d81c88c-cd00-4f72-80f1-78971a010ff1 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:40.974090 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-9d81c88c-cd00-4f72-80f1-78971a010ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:40.974090 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 798/1562] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:40.974090 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015bc47-774b-4ebd-a33d-0c538437bbbf None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:40.974090 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015bc47-774b-4ebd-a33d-0c538437bbbf None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:40.974090 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015bc47-774b-4ebd-a33d-0c538437bbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:40.993508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-f015bc47-774b-4ebd-a33d-0c538437bbbf None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:40.994710 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f015bc47-774b-4ebd-a33d-0c538437bbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:41.002421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 799/1563] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:41.003167 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015bc47-774b-4ebd-a33d-0c538437bbbf None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:41.013222 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015bc47-774b-4ebd-a33d-0c538437bbbf None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:41.018024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bcb2e-146b-46b5-8552-8fdaeabd03a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:41.018616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bcb2e-146b-46b5-8552-8fdaeabd03a9 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:41.018886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bcb2e-146b-46b5-8552-8fdaeabd03a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:41.019150 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bcb2e-146b-46b5-8552-8fdaeabd03a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:41.019873 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415bcb2e-146b-46b5-8552-8fdaeabd03a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:43:41.020200 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415bcb2e-146b-46b5-8552-8fdaeabd03a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:41.021810 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415bcb2e-146b-46b5-8552-8fdaeabd03a9 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:41.034031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 800/1564] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:43:41.036822 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bcb2e-146b-46b5-8552-8fdaeabd03a9 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:41.041184 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 765/1565] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:43:40 2022] POST /identity/v3/projects => generated 375 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:41.041184 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e8590-aa51-40a1-ac53-1a6a052b580b None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:41.041184 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e8590-aa51-40a1-ac53-1a6a052b580b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:41.041184 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e8590-aa51-40a1-ac53-1a6a052b580b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:41.052074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bcb2e-146b-46b5-8552-8fdaeabd03a9 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:41.060997 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-056e8590-aa51-40a1-ac53-1a6a052b580b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:41.062450 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-056e8590-aa51-40a1-ac53-1a6a052b580b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:41.110600 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc896df-a608-44ab-9c62-33e6b0748890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:41.111249 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc896df-a608-44ab-9c62-33e6b0748890 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:41.111553 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc896df-a608-44ab-9c62-33e6b0748890 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:41.111824 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc896df-a608-44ab-9c62-33e6b0748890 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:41.112483 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc896df-a608-44ab-9c62-33e6b0748890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:41.114411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc896df-a608-44ab-9c62-33e6b0748890 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:41.125789 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 766/1566] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:41.127050 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e271ccc-e3f0-437a-ac67-d2873db775ab None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:41.127335 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e271ccc-e3f0-437a-ac67-d2873db775ab None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:41.127606 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e271ccc-e3f0-437a-ac67-d2873db775ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:41.155972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-2e271ccc-e3f0-437a-ac67-d2873db775ab None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:41.157165 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-2e271ccc-e3f0-437a-ac67-d2873db775ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:41.165372 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 767/1567] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:41.166155 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e271ccc-e3f0-437a-ac67-d2873db775ab None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:41.175539 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e271ccc-e3f0-437a-ac67-d2873db775ab None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:41.182147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcf5f9-af11-4106-abb4-d3c3f8faa4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:41.183676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcf5f9-af11-4106-abb4-d3c3f8faa4cd admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:41.183676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcf5f9-af11-4106-abb4-d3c3f8faa4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:41.183676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcf5f9-af11-4106-abb4-d3c3f8faa4cd admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:41.184249 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcf5f9-af11-4106-abb4-d3c3f8faa4cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:43:41.184556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcf5f9-af11-4106-abb4-d3c3f8faa4cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:41.185637 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: WARNING py.warnings [None req-2bfcf5f9-af11-4106-abb4-d3c3f8faa4cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:43:41.185637 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: warnings.warn(msg) Aug 29 22:43:41.185637 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: Aug 29 22:43:41.186516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcf5f9-af11-4106-abb4-d3c3f8faa4cd admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:41.199208 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 768/1568] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:43:41.200395 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcf5f9-af11-4106-abb4-d3c3f8faa4cd admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:41.209873 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 801/1569] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:43:41 2022] POST /identity/v3/users => generated 283 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:41.209873 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc896df-a608-44ab-9c62-33e6b0748890 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:41.216839 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcf5f9-af11-4106-abb4-d3c3f8faa4cd admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:41.218727 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc896df-a608-44ab-9c62-33e6b0748890 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:41.221418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48baa297-9af7-4ebb-9929-15f0d7674c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:41.222054 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baa297-9af7-4ebb-9929-15f0d7674c09 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:41.222308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baa297-9af7-4ebb-9929-15f0d7674c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:41.222529 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baa297-9af7-4ebb-9929-15f0d7674c09 admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:41.223160 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48baa297-9af7-4ebb-9929-15f0d7674c09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:43:41.223457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48baa297-9af7-4ebb-9929-15f0d7674c09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:41.223859 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0c676-f382-47a1-aea4-0b7915084013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:41.224488 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0c676-f382-47a1-aea4-0b7915084013 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:41.224770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0c676-f382-47a1-aea4-0b7915084013 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:41.225040 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0c676-f382-47a1-aea4-0b7915084013 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:41.225138 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48baa297-9af7-4ebb-9929-15f0d7674c09 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:41.225685 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f0c676-f382-47a1-aea4-0b7915084013 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:43:41.225996 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f0c676-f382-47a1-aea4-0b7915084013 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:41.228096 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f0c676-f382-47a1-aea4-0b7915084013 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:41.237327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 769/1570] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:43:41.238312 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48baa297-9af7-4ebb-9929-15f0d7674c09 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:41.248176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 802/1571] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:43:41.248176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0c676-f382-47a1-aea4-0b7915084013 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:41.256300 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0c676-f382-47a1-aea4-0b7915084013 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:41.257167 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48baa297-9af7-4ebb-9929-15f0d7674c09 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:41.262057 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f1513-f213-49ee-9315-4a6aa3ccb9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:41.262608 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f1513-f213-49ee-9315-4a6aa3ccb9d9 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:41.262804 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ca66c-9047-4295-bdf4-3e22c0349f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:41.262950 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f1513-f213-49ee-9315-4a6aa3ccb9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:41.263139 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f1513-f213-49ee-9315-4a6aa3ccb9d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:41.263425 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ca66c-9047-4295-bdf4-3e22c0349f63 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:41.263695 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ca66c-9047-4295-bdf4-3e22c0349f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:41.263837 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f1513-f213-49ee-9315-4a6aa3ccb9d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:41.263961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ca66c-9047-4295-bdf4-3e22c0349f63 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:41.264614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ca66c-9047-4295-bdf4-3e22c0349f63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:41.264782 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: WARNING py.warnings [None req-432f1513-f213-49ee-9315-4a6aa3ccb9d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:43:41.264782 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: warnings.warn(msg) Aug 29 22:43:41.264782 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: Aug 29 22:43:41.265623 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: WARNING py.warnings [None req-6e7ca66c-9047-4295-bdf4-3e22c0349f63 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:43:41.265623 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: warnings.warn(msg) Aug 29 22:43:41.265623 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: Aug 29 22:43:41.266027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f1513-f213-49ee-9315-4a6aa3ccb9d9 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:41.266809 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ca66c-9047-4295-bdf4-3e22c0349f63 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:41.347043 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 770/1572] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:43:41 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:41.347043 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ca66c-9047-4295-bdf4-3e22c0349f63 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:41.347043 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6e7ca66c-9047-4295-bdf4-3e22c0349f63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:41.361244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 803/1573] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:43:41 2022] POST /identity/v3/projects => generated 363 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:41.361244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f1513-f213-49ee-9315-4a6aa3ccb9d9 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:41.388447 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:41.407857 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ca66c-9047-4295-bdf4-3e22c0349f63 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:41.430989 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f1513-f213-49ee-9315-4a6aa3ccb9d9 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:41.436353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949676c3-0b9c-49a4-96ab-f3cf72097780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:41.436972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949676c3-0b9c-49a4-96ab-f3cf72097780 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:41.437255 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949676c3-0b9c-49a4-96ab-f3cf72097780 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:41.437524 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949676c3-0b9c-49a4-96ab-f3cf72097780 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:41.438168 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949676c3-0b9c-49a4-96ab-f3cf72097780 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:43:41.438467 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949676c3-0b9c-49a4-96ab-f3cf72097780 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:41.440102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949676c3-0b9c-49a4-96ab-f3cf72097780 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:41.460074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 804/1574] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:43:41.460074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949676c3-0b9c-49a4-96ab-f3cf72097780 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:41.464119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74feb529-fde2-47e4-898c-244e62fce729 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:41.464763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74feb529-fde2-47e4-898c-244e62fce729 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:41.465046 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74feb529-fde2-47e4-898c-244e62fce729 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:41.465205 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949676c3-0b9c-49a4-96ab-f3cf72097780 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:41.465357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74feb529-fde2-47e4-898c-244e62fce729 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:41.470447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a8e47-226c-40e6-9451-7e54ee1eee25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:41.471204 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a8e47-226c-40e6-9451-7e54ee1eee25 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:41.471739 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a8e47-226c-40e6-9451-7e54ee1eee25 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:41.472331 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a8e47-226c-40e6-9451-7e54ee1eee25 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:41.473078 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a8e47-226c-40e6-9451-7e54ee1eee25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:41.475252 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a8e47-226c-40e6-9451-7e54ee1eee25 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:41.475896 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74feb529-fde2-47e4-898c-244e62fce729 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:41.478786 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74feb529-fde2-47e4-898c-244e62fce729 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:41.482387 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:41.489881 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 771/1575] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:43:41.490852 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74feb529-fde2-47e4-898c-244e62fce729 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:41.501109 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74feb529-fde2-47e4-898c-244e62fce729 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:41.506652 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1e4e3-eea9-426b-91db-a7d436c6ce77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:41.507277 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1e4e3-eea9-426b-91db-a7d436c6ce77 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:41.507539 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1e4e3-eea9-426b-91db-a7d436c6ce77 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:41.507808 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1e4e3-eea9-426b-91db-a7d436c6ce77 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:41.508693 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1e4e3-eea9-426b-91db-a7d436c6ce77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:41.510393 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1e4e3-eea9-426b-91db-a7d436c6ce77 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:41.520989 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 772/1576] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:41.521796 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1e4e3-eea9-426b-91db-a7d436c6ce77 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:41.533852 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1e4e3-eea9-426b-91db-a7d436c6ce77 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:41.538452 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58906014-c96a-4339-89ae-875f9274d99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:41.539042 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58906014-c96a-4339-89ae-875f9274d99f admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:41.539317 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58906014-c96a-4339-89ae-875f9274d99f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:41.540214 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58906014-c96a-4339-89ae-875f9274d99f admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:41.540844 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58906014-c96a-4339-89ae-875f9274d99f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:41.542770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58906014-c96a-4339-89ae-875f9274d99f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:41.603359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 805/1577] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:43:41 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:41.604216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a8e47-226c-40e6-9451-7e54ee1eee25 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:41.645389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 773/1578] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:43:41 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:41.652780 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58906014-c96a-4339-89ae-875f9274d99f admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:41.672846 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a8e47-226c-40e6-9451-7e54ee1eee25 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:41.718614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58906014-c96a-4339-89ae-875f9274d99f admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:41.723624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41313e11-4a77-42ba-8871-6b07e814ae2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:41.724343 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41313e11-4a77-42ba-8871-6b07e814ae2f admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:41.724624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41313e11-4a77-42ba-8871-6b07e814ae2f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:41.729879 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41313e11-4a77-42ba-8871-6b07e814ae2f admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:41.729879 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41313e11-4a77-42ba-8871-6b07e814ae2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:41.729879 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41313e11-4a77-42ba-8871-6b07e814ae2f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:41.740562 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 774/1579] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:41.741392 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41313e11-4a77-42ba-8871-6b07e814ae2f admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:41.741912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e959af-8dc7-486f-a058-045181092f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:41.742502 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e959af-8dc7-486f-a058-045181092f80 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:41.742786 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e959af-8dc7-486f-a058-045181092f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:41.743065 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e959af-8dc7-486f-a058-045181092f80 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:41.744116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e959af-8dc7-486f-a058-045181092f80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:41.746126 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e959af-8dc7-486f-a058-045181092f80 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:41.751385 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41313e11-4a77-42ba-8871-6b07e814ae2f admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:41.756564 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc760cc-0ccf-4d89-a565-e99a57093ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:41.757754 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc760cc-0ccf-4d89-a565-e99a57093ef8 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:41.757754 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc760cc-0ccf-4d89-a565-e99a57093ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:41.757754 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc760cc-0ccf-4d89-a565-e99a57093ef8 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:41.758438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc760cc-0ccf-4d89-a565-e99a57093ef8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:41.760454 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc760cc-0ccf-4d89-a565-e99a57093ef8 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:41.872889 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 806/1580] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:43:41 2022] POST /identity/v3/projects => generated 367 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:41.873833 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e959af-8dc7-486f-a058-045181092f80 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:41.878151 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 775/1581] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:43:41 2022] POST /identity/v3/users => generated 282 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:41.881182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc760cc-0ccf-4d89-a565-e99a57093ef8 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:41.915343 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc760cc-0ccf-4d89-a565-e99a57093ef8 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:41.943751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:41.957469 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e959af-8dc7-486f-a058-045181092f80 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:41.962933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169cfe74-4fe4-4b69-b24e-50bd318ed7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:41.963542 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169cfe74-4fe4-4b69-b24e-50bd318ed7c8 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:41.963820 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169cfe74-4fe4-4b69-b24e-50bd318ed7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:41.964079 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169cfe74-4fe4-4b69-b24e-50bd318ed7c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:41.964495 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01034563-531b-4f19-bf30-6d966bb0954f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:41.964736 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169cfe74-4fe4-4b69-b24e-50bd318ed7c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:41.965270 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01034563-531b-4f19-bf30-6d966bb0954f admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:41.965582 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01034563-531b-4f19-bf30-6d966bb0954f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:41.965913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01034563-531b-4f19-bf30-6d966bb0954f admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:41.966616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169cfe74-4fe4-4b69-b24e-50bd318ed7c8 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:41.966739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01034563-531b-4f19-bf30-6d966bb0954f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:41.968487 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01034563-531b-4f19-bf30-6d966bb0954f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:41.980243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 776/1582] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:41.982798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01034563-531b-4f19-bf30-6d966bb0954f admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:41.991387 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01034563-531b-4f19-bf30-6d966bb0954f admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:41.996921 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998ef0f-fc9b-42d9-89f1-f251f1342d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:41.997505 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998ef0f-fc9b-42d9-89f1-f251f1342d67 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:41.997783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998ef0f-fc9b-42d9-89f1-f251f1342d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:41.998054 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998ef0f-fc9b-42d9-89f1-f251f1342d67 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:41.998677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a998ef0f-fc9b-42d9-89f1-f251f1342d67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:42.000706 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a998ef0f-fc9b-42d9-89f1-f251f1342d67 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:42.069516 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 807/1583] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:43:41 2022] POST /identity/v3/projects => generated 391 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:42.069516 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169cfe74-4fe4-4b69-b24e-50bd318ed7c8 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:42.070513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 777/1584] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:43:41 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:42.072285 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998ef0f-fc9b-42d9-89f1-f251f1342d67 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:42.077397 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:42.082820 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998ef0f-fc9b-42d9-89f1-f251f1342d67 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:42.141093 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169cfe74-4fe4-4b69-b24e-50bd318ed7c8 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:42.146947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952cd295-0e63-48fa-b0f2-897d2709bfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:42.147566 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cd295-0e63-48fa-b0f2-897d2709bfa0 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.147862 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cd295-0e63-48fa-b0f2-897d2709bfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.148151 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cd295-0e63-48fa-b0f2-897d2709bfa0 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:42.149619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952cd295-0e63-48fa-b0f2-897d2709bfa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:42.151673 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952cd295-0e63-48fa-b0f2-897d2709bfa0 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:42.154956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585f48b-507d-4769-a2b6-40d2d1fa5480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:42.155595 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585f48b-507d-4769-a2b6-40d2d1fa5480 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.155876 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585f48b-507d-4769-a2b6-40d2d1fa5480 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.156164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585f48b-507d-4769-a2b6-40d2d1fa5480 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:42.156816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585f48b-507d-4769-a2b6-40d2d1fa5480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:42.158507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585f48b-507d-4769-a2b6-40d2d1fa5480 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:42.169307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 778/1585] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:42.170224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585f48b-507d-4769-a2b6-40d2d1fa5480 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:42.180763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585f48b-507d-4769-a2b6-40d2d1fa5480 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:42.185375 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8217697-449e-469d-b6de-fd57bfbd94ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:42.185986 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8217697-449e-469d-b6de-fd57bfbd94ea admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.186264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8217697-449e-469d-b6de-fd57bfbd94ea admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.186531 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8217697-449e-469d-b6de-fd57bfbd94ea admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:42.187161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8217697-449e-469d-b6de-fd57bfbd94ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:42.188871 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8217697-449e-469d-b6de-fd57bfbd94ea admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:42.199810 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 779/1586] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:42.200635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8217697-449e-469d-b6de-fd57bfbd94ea admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:42.210338 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8217697-449e-469d-b6de-fd57bfbd94ea admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:42.216166 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785b67c-664c-43c9-81da-6f3caa5745bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:42.216836 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785b67c-664c-43c9-81da-6f3caa5745bb admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.217110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785b67c-664c-43c9-81da-6f3caa5745bb admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.217378 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785b67c-664c-43c9-81da-6f3caa5745bb admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:42.217998 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785b67c-664c-43c9-81da-6f3caa5745bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:42.219894 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785b67c-664c-43c9-81da-6f3caa5745bb admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:42.257199 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 808/1587] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:43:42 2022] POST /identity/v3/users => generated 290 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:42.257199 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952cd295-0e63-48fa-b0f2-897d2709bfa0 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:42.265291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952cd295-0e63-48fa-b0f2-897d2709bfa0 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:42.269870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f8cbf-ba48-452e-a59b-46afcdb63c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:42.270500 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f8cbf-ba48-452e-a59b-46afcdb63c31 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.270784 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f8cbf-ba48-452e-a59b-46afcdb63c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.271081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f8cbf-ba48-452e-a59b-46afcdb63c31 admin admin] PATH_INFO: `/v3/projects/e221944c64db423caec3b03179adf454/users/6223d34100414e49af647ddb4406c3cd/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:42.286718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f8cbf-ba48-452e-a59b-46afcdb63c31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e221944c64db423caec3b03179adf454, user_id=6223d34100414e49af647ddb4406c3cd, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:42.291538 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f8cbf-ba48-452e-a59b-46afcdb63c31 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:42.327803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 780/1588] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:43:42 2022] POST /identity/v3/users => generated 284 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:42.328724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785b67c-664c-43c9-81da-6f3caa5745bb admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:42.339671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785b67c-664c-43c9-81da-6f3caa5745bb admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:42.344425 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff700db-0427-4a30-9229-45dd7f563747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:42.345007 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff700db-0427-4a30-9229-45dd7f563747 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.345305 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff700db-0427-4a30-9229-45dd7f563747 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.345550 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff700db-0427-4a30-9229-45dd7f563747 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:42.346167 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff700db-0427-4a30-9229-45dd7f563747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:42.348095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff700db-0427-4a30-9229-45dd7f563747 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:42.430606 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 809/1589] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:43:42 2022] PUT /identity/v3/projects/e221944c64db423caec3b03179adf454/users/6223d34100414e49af647ddb4406c3cd/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:42.431845 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f8cbf-ba48-452e-a59b-46afcdb63c31 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:42.459151 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 781/1590] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:43:42 2022] POST /identity/v3/users => generated 296 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:42.460324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff700db-0427-4a30-9229-45dd7f563747 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:42.472006 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff700db-0427-4a30-9229-45dd7f563747 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:42.515649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f8cbf-ba48-452e-a59b-46afcdb63c31 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:42.521177 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c488e-f061-4fc5-b85b-c670f5ca6a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:42.522016 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c488e-f061-4fc5-b85b-c670f5ca6a04 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.522508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c488e-f061-4fc5-b85b-c670f5ca6a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.522967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c488e-f061-4fc5-b85b-c670f5ca6a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:42.523992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c488e-f061-4fc5-b85b-c670f5ca6a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:42.525829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c488e-f061-4fc5-b85b-c670f5ca6a04 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:42.536644 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 810/1591] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:42.537479 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c488e-f061-4fc5-b85b-c670f5ca6a04 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:42.537772 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3fd74-776c-4bb2-9b8f-68ceb9846ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:42.538582 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3fd74-776c-4bb2-9b8f-68ceb9846ea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.538855 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3fd74-776c-4bb2-9b8f-68ceb9846ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.539119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3fd74-776c-4bb2-9b8f-68ceb9846ea1 admin admin] PATH_INFO: `/v3/projects/df7ab727f89e4fcb9933c2098b4fb501/users/43cdaa09925c4b00be3b45cea4fd40b7/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:42.549254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c488e-f061-4fc5-b85b-c670f5ca6a04 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:42.554106 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d495739-4a3d-41a1-b413-8924dfa5b0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:42.554567 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:42.554730 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d495739-4a3d-41a1-b413-8924dfa5b0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.554969 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d495739-4a3d-41a1-b413-8924dfa5b0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.556027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d495739-4a3d-41a1-b413-8924dfa5b0f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:42.556764 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d495739-4a3d-41a1-b413-8924dfa5b0f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:42.558373 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d495739-4a3d-41a1-b413-8924dfa5b0f2 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:42.561049 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3fd74-776c-4bb2-9b8f-68ceb9846ea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df7ab727f89e4fcb9933c2098b4fb501, user_id=43cdaa09925c4b00be3b45cea4fd40b7, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:42.561771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3fd74-776c-4bb2-9b8f-68ceb9846ea1 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:42.568745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 811/1592] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:42.569702 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d495739-4a3d-41a1-b413-8924dfa5b0f2 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:42.580205 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d495739-4a3d-41a1-b413-8924dfa5b0f2 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:42.585300 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810f040-b6b6-4de9-8d29-9fb52725257a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:42.586131 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810f040-b6b6-4de9-8d29-9fb52725257a admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.586448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810f040-b6b6-4de9-8d29-9fb52725257a admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.586722 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810f040-b6b6-4de9-8d29-9fb52725257a admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:42.587338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9810f040-b6b6-4de9-8d29-9fb52725257a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:42.588978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9810f040-b6b6-4de9-8d29-9fb52725257a admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:42.599509 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 812/1593] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:42.605053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810f040-b6b6-4de9-8d29-9fb52725257a admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:42.609916 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810f040-b6b6-4de9-8d29-9fb52725257a admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:42.614690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e98f2-3377-4e2f-b224-df83cdd45581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:42.615384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e98f2-3377-4e2f-b224-df83cdd45581 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.615594 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e98f2-3377-4e2f-b224-df83cdd45581 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.615862 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e98f2-3377-4e2f-b224-df83cdd45581 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:42.616621 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e98f2-3377-4e2f-b224-df83cdd45581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:42.619178 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e98f2-3377-4e2f-b224-df83cdd45581 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:42.629528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 813/1594] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:42.630713 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c03fcd-3a6c-4906-8ed1-302a232d228a None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.630991 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c03fcd-3a6c-4906-8ed1-302a232d228a None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.631254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c03fcd-3a6c-4906-8ed1-302a232d228a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:42.660501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-e6c03fcd-3a6c-4906-8ed1-302a232d228a None None] MFA Rules not processed for user `6223d34100414e49af647ddb4406c3cd`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:42.661701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e6c03fcd-3a6c-4906-8ed1-302a232d228a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:42.662661 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 782/1595] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:43:42 2022] PUT /identity/v3/projects/df7ab727f89e4fcb9933c2098b4fb501/users/43cdaa09925c4b00be3b45cea4fd40b7/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:42.664388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3fd74-776c-4bb2-9b8f-68ceb9846ea1 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:42.688581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:42.734866 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3fd74-776c-4bb2-9b8f-68ceb9846ea1 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:42.739723 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da98e8-fc15-4e07-aa9c-422854d92eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:42.740396 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da98e8-fc15-4e07-aa9c-422854d92eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.740682 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da98e8-fc15-4e07-aa9c-422854d92eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.740948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da98e8-fc15-4e07-aa9c-422854d92eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:42.741581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da98e8-fc15-4e07-aa9c-422854d92eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:42.743687 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 814/1596] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:42.743886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da98e8-fc15-4e07-aa9c-422854d92eb8 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:42.744618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c03fcd-3a6c-4906-8ed1-302a232d228a None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:42.759166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c03fcd-3a6c-4906-8ed1-302a232d228a None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:42.759718 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 783/1597] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:42.760650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da98e8-fc15-4e07-aa9c-422854d92eb8 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:42.766712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3aca8-c6d6-453c-8be1-19a3eb961604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:42.767403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3aca8-c6d6-453c-8be1-19a3eb961604 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.767666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3aca8-c6d6-453c-8be1-19a3eb961604 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.767953 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3aca8-c6d6-453c-8be1-19a3eb961604 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:42.769101 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3aca8-c6d6-453c-8be1-19a3eb961604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:42.770348 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3aca8-c6d6-453c-8be1-19a3eb961604 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:42.771441 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da98e8-fc15-4e07-aa9c-422854d92eb8 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:42.776321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0235df-1a29-4f73-954c-5d20924dcfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:42.779735 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0235df-1a29-4f73-954c-5d20924dcfb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.779735 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0235df-1a29-4f73-954c-5d20924dcfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.779735 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0235df-1a29-4f73-954c-5d20924dcfb4 admin admin] PATH_INFO: `/v3/projects/f27cc7902bb04528b9b9b899bcd25206/users/c92e8f3339c44c5bb2a3a385d42d4f68/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:42.781836 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 815/1598] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:42.782943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3aca8-c6d6-453c-8be1-19a3eb961604 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:42.793792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3aca8-c6d6-453c-8be1-19a3eb961604 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:42.797979 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0235df-1a29-4f73-954c-5d20924dcfb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f27cc7902bb04528b9b9b899bcd25206, user_id=c92e8f3339c44c5bb2a3a385d42d4f68, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:42.797979 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0235df-1a29-4f73-954c-5d20924dcfb4 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:42.798848 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd79fb6-2d1b-4bc5-808a-cdccf7aebc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:42.799505 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd79fb6-2d1b-4bc5-808a-cdccf7aebc1e admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.799666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd79fb6-2d1b-4bc5-808a-cdccf7aebc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.799879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd79fb6-2d1b-4bc5-808a-cdccf7aebc1e admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:42.800564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd79fb6-2d1b-4bc5-808a-cdccf7aebc1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:42.802145 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd79fb6-2d1b-4bc5-808a-cdccf7aebc1e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:42.813332 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 816/1599] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:42.814144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd79fb6-2d1b-4bc5-808a-cdccf7aebc1e admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:42.824393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd79fb6-2d1b-4bc5-808a-cdccf7aebc1e admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:42.829203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0441c-51c6-4c46-af97-ad7bdac74e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:42.829793 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0441c-51c6-4c46-af97-ad7bdac74e5b admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.830072 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0441c-51c6-4c46-af97-ad7bdac74e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.830375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0441c-51c6-4c46-af97-ad7bdac74e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:42.831028 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0441c-51c6-4c46-af97-ad7bdac74e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:42.832646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0441c-51c6-4c46-af97-ad7bdac74e5b admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:42.843673 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 817/1600] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:42.845492 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309c233-2d02-4f63-a4d3-30e594332ba1 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.845492 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309c233-2d02-4f63-a4d3-30e594332ba1 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.845492 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309c233-2d02-4f63-a4d3-30e594332ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:42.865936 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-c309c233-2d02-4f63-a4d3-30e594332ba1 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:42.867144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c309c233-2d02-4f63-a4d3-30e594332ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:42.876285 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 784/1601] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:43:42 2022] PUT /identity/v3/projects/f27cc7902bb04528b9b9b899bcd25206/users/c92e8f3339c44c5bb2a3a385d42d4f68/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:42.878187 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0235df-1a29-4f73-954c-5d20924dcfb4 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:42.948157 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 818/1602] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:42.948157 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309c233-2d02-4f63-a4d3-30e594332ba1 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:42.956013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309c233-2d02-4f63-a4d3-30e594332ba1 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:42.960028 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0235df-1a29-4f73-954c-5d20924dcfb4 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:42.961127 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4731f1ef-93a1-4f2a-81a1-fb7b35567e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:42.962508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731f1ef-93a1-4f2a-81a1-fb7b35567e5f admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.963031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731f1ef-93a1-4f2a-81a1-fb7b35567e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.963310 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731f1ef-93a1-4f2a-81a1-fb7b35567e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:42.963960 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4731f1ef-93a1-4f2a-81a1-fb7b35567e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:42.965596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4731f1ef-93a1-4f2a-81a1-fb7b35567e5f admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:42.966571 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1fc03-feb8-4f46-b46f-7124836bf061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:42.967296 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1fc03-feb8-4f46-b46f-7124836bf061 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.967713 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1fc03-feb8-4f46-b46f-7124836bf061 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.968088 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1fc03-feb8-4f46-b46f-7124836bf061 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:42.968840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1fc03-feb8-4f46-b46f-7124836bf061 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:42.971047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1fc03-feb8-4f46-b46f-7124836bf061 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:42.976472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 819/1603] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:42.977400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4731f1ef-93a1-4f2a-81a1-fb7b35567e5f admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:42.987347 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4731f1ef-93a1-4f2a-81a1-fb7b35567e5f admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:42.993143 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fcf585-0774-40e7-beb7-7e4c80b7145e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:42.994033 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcf585-0774-40e7-beb7-7e4c80b7145e admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:42.994352 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcf585-0774-40e7-beb7-7e4c80b7145e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:42.994580 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcf585-0774-40e7-beb7-7e4c80b7145e admin admin] PATH_INFO: `/v3/projects/8ba704574496408d84b349f1c44ecc80/users/75eb21cb023f482ca5cba476a315a32f/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:43.011497 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fcf585-0774-40e7-beb7-7e4c80b7145e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba704574496408d84b349f1c44ecc80, user_id=75eb21cb023f482ca5cba476a315a32f, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:43.014430 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fcf585-0774-40e7-beb7-7e4c80b7145e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:43.089745 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 785/1604] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:43:42 2022] POST /identity/v3/projects => generated 363 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:43.091113 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1fc03-feb8-4f46-b46f-7124836bf061 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:43.149350 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 820/1605] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:43:42 2022] PUT /identity/v3/projects/8ba704574496408d84b349f1c44ecc80/users/75eb21cb023f482ca5cba476a315a32f/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:43.149350 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fcf585-0774-40e7-beb7-7e4c80b7145e admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:43.161100 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:43.181951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1fc03-feb8-4f46-b46f-7124836bf061 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:43.259217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2cae9-8691-40ba-9dbd-523115497988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:43.259782 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2cae9-8691-40ba-9dbd-523115497988 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:43.260054 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2cae9-8691-40ba-9dbd-523115497988 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:43.260354 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2cae9-8691-40ba-9dbd-523115497988 admin admin] PATH_INFO: `/v3/projects/844009bb229a4357bb183c22473ec045/users/b21dc8a79bce4b26a690d8990c52116c/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:43.265503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:43.276270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fcf585-0774-40e7-beb7-7e4c80b7145e admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:43.279618 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2cae9-8691-40ba-9dbd-523115497988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=844009bb229a4357bb183c22473ec045, user_id=b21dc8a79bce4b26a690d8990c52116c, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:43.282364 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2cae9-8691-40ba-9dbd-523115497988 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:43.284195 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0c537-5008-456f-804b-96d24a78eef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:43.285530 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0c537-5008-456f-804b-96d24a78eef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:43.286106 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0c537-5008-456f-804b-96d24a78eef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:43.286665 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0c537-5008-456f-804b-96d24a78eef7 admin admin] PATH_INFO: `/v3/projects/63c73254461e455e9a3971730dc6e0d5/users/1917db952a4047cebbe6d1b232996e6d/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:43.312660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0c537-5008-456f-804b-96d24a78eef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63c73254461e455e9a3971730dc6e0d5, user_id=1917db952a4047cebbe6d1b232996e6d, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:43.318091 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0c537-5008-456f-804b-96d24a78eef7 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:43.408197 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 786/1607] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:43:43 2022] PUT /identity/v3/projects/844009bb229a4357bb183c22473ec045/users/b21dc8a79bce4b26a690d8990c52116c/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:43.410692 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72597a-bc3e-4ee8-9874-29fe82eac08e None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:43.411235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72597a-bc3e-4ee8-9874-29fe82eac08e None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:43.411726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72597a-bc3e-4ee8-9874-29fe82eac08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:43.413081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 821/1607] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:43:43 2022] PUT /identity/v3/projects/63c73254461e455e9a3971730dc6e0d5/users/1917db952a4047cebbe6d1b232996e6d/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:43.414108 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0c537-5008-456f-804b-96d24a78eef7 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:43.459208 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-cb72597a-bc3e-4ee8-9874-29fe82eac08e None None] MFA Rules not processed for user `c92e8f3339c44c5bb2a3a385d42d4f68`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:43.465643 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-cb72597a-bc3e-4ee8-9874-29fe82eac08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:43.479230 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0c537-5008-456f-804b-96d24a78eef7 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:43.535041 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9ed5f-5f2c-447d-8dad-53646c2cd89d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:43.535651 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9ed5f-5f2c-447d-8dad-53646c2cd89d service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:43.535923 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9ed5f-5f2c-447d-8dad-53646c2cd89d service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:43.536219 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9ed5f-5f2c-447d-8dad-53646c2cd89d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:43.553391 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 787/1608] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:43.553391 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72597a-bc3e-4ee8-9874-29fe82eac08e None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:43.607938 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a9ed5f-5f2c-447d-8dad-53646c2cd89d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:43.609702 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a9ed5f-5f2c-447d-8dad-53646c2cd89d service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:43.623211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 822/1609] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:43:43.623211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9ed5f-5f2c-447d-8dad-53646c2cd89d service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:43.635341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72597a-bc3e-4ee8-9874-29fe82eac08e None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:43.640466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f06ca-7fb3-4798-998e-4e45f4562580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:43.641149 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f06ca-7fb3-4798-998e-4e45f4562580 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:43.641429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f06ca-7fb3-4798-998e-4e45f4562580 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:43.641677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9ed5f-5f2c-447d-8dad-53646c2cd89d service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:43.641808 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f06ca-7fb3-4798-998e-4e45f4562580 admin admin] PATH_INFO: `/v3/projects/66de012537ed40e3bc233b80458bcfc4/users/cc2f5c40aedf47dcb6374d64fcb04add/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:43.648072 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e9fdc-623e-442b-8e98-3dc36b983d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:43.649298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e9fdc-623e-442b-8e98-3dc36b983d5b admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:43.649701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e9fdc-623e-442b-8e98-3dc36b983d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:43.650131 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e9fdc-623e-442b-8e98-3dc36b983d5b admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:43.650873 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e9fdc-623e-442b-8e98-3dc36b983d5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:43.653103 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e9fdc-623e-442b-8e98-3dc36b983d5b admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:43.656716 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:43.661787 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f06ca-7fb3-4798-998e-4e45f4562580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66de012537ed40e3bc233b80458bcfc4, user_id=cc2f5c40aedf47dcb6374d64fcb04add, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:43.665785 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f06ca-7fb3-4798-998e-4e45f4562580 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:43.737172 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 823/1611] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:43:43 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:43.739113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cde27e-e61c-4da5-a16b-10ecfd1a12e3 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:43.739410 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 788/1611] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:43:43 2022] PUT /identity/v3/projects/66de012537ed40e3bc233b80458bcfc4/users/cc2f5c40aedf47dcb6374d64fcb04add/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:43.740023 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cde27e-e61c-4da5-a16b-10ecfd1a12e3 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:43.740543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cde27e-e61c-4da5-a16b-10ecfd1a12e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:43.741136 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0428e3e-a4d5-4b3d-a8e4-49b0e85397c5 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:43.741444 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0428e3e-a4d5-4b3d-a8e4-49b0e85397c5 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:43.741778 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0428e3e-a4d5-4b3d-a8e4-49b0e85397c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:43.769972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-c0428e3e-a4d5-4b3d-a8e4-49b0e85397c5 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:43.771822 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-c0428e3e-a4d5-4b3d-a8e4-49b0e85397c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:43.784750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-f3cde27e-e61c-4da5-a16b-10ecfd1a12e3 None None] MFA Rules not processed for user `75eb21cb023f482ca5cba476a315a32f`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:43.784750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f3cde27e-e61c-4da5-a16b-10ecfd1a12e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:43.803200 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:43.846784 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 789/1612] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:43.848789 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519310d-29e6-4fce-856b-a752bf900dae None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:43.859791 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519310d-29e6-4fce-856b-a752bf900dae None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:43.859791 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519310d-29e6-4fce-856b-a752bf900dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:43.877976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 824/1613] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:43.879284 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cde27e-e61c-4da5-a16b-10ecfd1a12e3 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:43.894274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cde27e-e61c-4da5-a16b-10ecfd1a12e3 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:43.899702 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870e330-924e-406f-888e-7345dc104ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:43.900361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870e330-924e-406f-888e-7345dc104ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:43.900635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870e330-924e-406f-888e-7345dc104ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:43.900900 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870e330-924e-406f-888e-7345dc104ca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:43.901534 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870e330-924e-406f-888e-7345dc104ca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:43.903674 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870e330-924e-406f-888e-7345dc104ca5 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:43.944442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-5519310d-29e6-4fce-856b-a752bf900dae None None] MFA Rules not processed for user `1917db952a4047cebbe6d1b232996e6d`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:43.946277 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-5519310d-29e6-4fce-856b-a752bf900dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:44.004577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 825/1614] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:43:43 2022] POST /identity/v3/projects => generated 363 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:44.006632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dbd46-a4ec-4c68-a5f1-7fc7ac203cb8 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:44.007298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dbd46-a4ec-4c68-a5f1-7fc7ac203cb8 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:44.007682 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dbd46-a4ec-4c68-a5f1-7fc7ac203cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:44.036375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-496dbd46-a4ec-4c68-a5f1-7fc7ac203cb8 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:44.037763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-496dbd46-a4ec-4c68-a5f1-7fc7ac203cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:44.044849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 790/1615] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:44.045864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519310d-29e6-4fce-856b-a752bf900dae None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:44.060312 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519310d-29e6-4fce-856b-a752bf900dae None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:44.140624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 826/1616] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:44.144288 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee9c1f-48eb-43c8-93ac-68629e237078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:44.144441 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496dbd46-a4ec-4c68-a5f1-7fc7ac203cb8 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:44.144637 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee9c1f-48eb-43c8-93ac-68629e237078 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:44.144637 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee9c1f-48eb-43c8-93ac-68629e237078 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:44.144637 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee9c1f-48eb-43c8-93ac-68629e237078 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:44.144637 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee9c1f-48eb-43c8-93ac-68629e237078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:44.145560 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee9c1f-48eb-43c8-93ac-68629e237078 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:44.163827 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 791/1617] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:44.166691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd75676-4a35-45ef-990c-7bd6e8d64a8e None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:44.167022 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd75676-4a35-45ef-990c-7bd6e8d64a8e None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:44.167303 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd75676-4a35-45ef-990c-7bd6e8d64a8e None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:44.173605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 792/1618] 10.222.0.59 () {54 vars in 962 bytes} [Mon Aug 29 22:43:44 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:43:44.181542 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd75676-4a35-45ef-990c-7bd6e8d64a8e None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:44.290783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd75676-4a35-45ef-990c-7bd6e8d64a8e None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:44.290783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813551cc-e39c-424f-9b67-7de3003f7f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:44.290783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813551cc-e39c-424f-9b67-7de3003f7f9a admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:44.290783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813551cc-e39c-424f-9b67-7de3003f7f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:44.291699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813551cc-e39c-424f-9b67-7de3003f7f9a admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:44.291699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813551cc-e39c-424f-9b67-7de3003f7f9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:44.291699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813551cc-e39c-424f-9b67-7de3003f7f9a admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:44.292336 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496dbd46-a4ec-4c68-a5f1-7fc7ac203cb8 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:44.292336 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca36930-8996-4784-8a52-593af2c7a212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:44.292336 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca36930-8996-4784-8a52-593af2c7a212 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:44.292336 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca36930-8996-4784-8a52-593af2c7a212 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:44.293616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca36930-8996-4784-8a52-593af2c7a212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:44.293616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca36930-8996-4784-8a52-593af2c7a212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:44.293616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca36930-8996-4784-8a52-593af2c7a212 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:44.293616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 827/1619] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:43:44.293616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca36930-8996-4784-8a52-593af2c7a212 service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:44.302212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca36930-8996-4784-8a52-593af2c7a212 service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:44.308366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e645e2-c7a9-4a5e-9265-de79db3ad7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:44.308569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e645e2-c7a9-4a5e-9265-de79db3ad7f4 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:44.308822 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e645e2-c7a9-4a5e-9265-de79db3ad7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:44.309182 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e645e2-c7a9-4a5e-9265-de79db3ad7f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:44.309940 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e645e2-c7a9-4a5e-9265-de79db3ad7f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:44.311886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e645e2-c7a9-4a5e-9265-de79db3ad7f4 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:44.351170 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:44.366556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 793/1620] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:43:44 2022] POST /identity/v3/users => generated 288 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:44.371368 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813551cc-e39c-424f-9b67-7de3003f7f9a admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:44.387671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813551cc-e39c-424f-9b67-7de3003f7f9a admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:44.393180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee9591-6117-4a24-960a-149b6ad540b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:44.397255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee9591-6117-4a24-960a-149b6ad540b2 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:44.397255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee9591-6117-4a24-960a-149b6ad540b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:44.397255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee9591-6117-4a24-960a-149b6ad540b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:44.405319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee9591-6117-4a24-960a-149b6ad540b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:44.406815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee9591-6117-4a24-960a-149b6ad540b2 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:44.417082 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 794/1621] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:43:44.417910 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee9591-6117-4a24-960a-149b6ad540b2 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:44.433247 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee9591-6117-4a24-960a-149b6ad540b2 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:44.435579 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 828/1622] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:43:44 2022] POST /identity/v3/projects => generated 367 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:44.437184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254194b5-9ad0-40c1-967b-2e5cf03eb8ef None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:44.437560 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254194b5-9ad0-40c1-967b-2e5cf03eb8ef None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:44.437840 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254194b5-9ad0-40c1-967b-2e5cf03eb8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:44.481324 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:44.492733 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-254194b5-9ad0-40c1-967b-2e5cf03eb8ef None None] MFA Rules not processed for user `7f21105679f245a19ea4850e6c010dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:44.494828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-254194b5-9ad0-40c1-967b-2e5cf03eb8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:44.518459 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681e27e-d347-4bde-9388-425500842afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:44.519254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681e27e-d347-4bde-9388-425500842afb admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:44.519573 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681e27e-d347-4bde-9388-425500842afb admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:44.519851 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681e27e-d347-4bde-9388-425500842afb admin admin] PATH_INFO: `/v3/projects/4fa71891cd6f41a89cb770b5d1c47b8b/users/1d0ef9839e2a49ca9221640b2f9588ef/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:44.537557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681e27e-d347-4bde-9388-425500842afb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa71891cd6f41a89cb770b5d1c47b8b, user_id=1d0ef9839e2a49ca9221640b2f9588ef, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:44.541607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681e27e-d347-4bde-9388-425500842afb admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:44.588201 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 829/1623] 10.222.0.59 () {58 vars in 1059 bytes} [Mon Aug 29 22:43:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:44.589688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254194b5-9ad0-40c1-967b-2e5cf03eb8ef None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:44.602642 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254194b5-9ad0-40c1-967b-2e5cf03eb8ef None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:44.609151 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f186a5-4cc1-4c0a-9f35-b793ac51e230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:44.614398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f186a5-4cc1-4c0a-9f35-b793ac51e230 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:44.614398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f186a5-4cc1-4c0a-9f35-b793ac51e230 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:44.614398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f186a5-4cc1-4c0a-9f35-b793ac51e230 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:44.614398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f186a5-4cc1-4c0a-9f35-b793ac51e230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:44.614398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f186a5-4cc1-4c0a-9f35-b793ac51e230 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:44.627263 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 830/1624] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:44.628347 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f186a5-4cc1-4c0a-9f35-b793ac51e230 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:44.645116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f186a5-4cc1-4c0a-9f35-b793ac51e230 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:44.649508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 795/1625] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:43:44 2022] PUT /identity/v3/projects/4fa71891cd6f41a89cb770b5d1c47b8b/users/1d0ef9839e2a49ca9221640b2f9588ef/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:44.649508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681e27e-d347-4bde-9388-425500842afb admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:44.718838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd61b76-5b7a-4af2-a96a-145f4d1de083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:44.719892 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd61b76-5b7a-4af2-a96a-145f4d1de083 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:44.720379 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd61b76-5b7a-4af2-a96a-145f4d1de083 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:44.720868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd61b76-5b7a-4af2-a96a-145f4d1de083 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:44.721800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd61b76-5b7a-4af2-a96a-145f4d1de083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:44.724127 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd61b76-5b7a-4af2-a96a-145f4d1de083 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:44.763706 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681e27e-d347-4bde-9388-425500842afb admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:44.772216 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6ee0b-a691-462e-90b0-fbaa3b092097 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:44.773031 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6ee0b-a691-462e-90b0-fbaa3b092097 service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:44.773315 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6ee0b-a691-462e-90b0-fbaa3b092097 service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:44.774624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6ee0b-a691-462e-90b0-fbaa3b092097 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:44.866855 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a6ee0b-a691-462e-90b0-fbaa3b092097 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:44.866855 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a6ee0b-a691-462e-90b0-fbaa3b092097 service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:44.882399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 796/1626] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:43:44.886586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6ee0b-a691-462e-90b0-fbaa3b092097 service nova] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:44.907143 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6ee0b-a691-462e-90b0-fbaa3b092097 service nova] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:44.927328 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb75f5-2072-48e5-a333-f2444b1f3b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:44.941367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb75f5-2072-48e5-a333-f2444b1f3b19 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:44.941367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb75f5-2072-48e5-a333-f2444b1f3b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:44.941367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb75f5-2072-48e5-a333-f2444b1f3b19 admin admin] PATH_INFO: `/v3/projects/07f5bd35d90f425280b542cdeae03139/users/16825cf32f494c6b97776ff1ba38ec4b/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:44.941764 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 831/1627] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:43:44 2022] POST /identity/v3/users => generated 290 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:44.941764 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd61b76-5b7a-4af2-a96a-145f4d1de083 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:44.953387 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd61b76-5b7a-4af2-a96a-145f4d1de083 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:44.957005 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb75f5-2072-48e5-a333-f2444b1f3b19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07f5bd35d90f425280b542cdeae03139, user_id=16825cf32f494c6b97776ff1ba38ec4b, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:44.962103 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb75f5-2072-48e5-a333-f2444b1f3b19 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:44.981328 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d12597e-209a-424c-b935-2458e4c43373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:44.982492 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12597e-209a-424c-b935-2458e4c43373 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:44.983037 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12597e-209a-424c-b935-2458e4c43373 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:44.984232 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12597e-209a-424c-b935-2458e4c43373 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:44.985460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d12597e-209a-424c-b935-2458e4c43373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:44.987061 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d12597e-209a-424c-b935-2458e4c43373 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:45.012282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 832/1628] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:45.013157 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d12597e-209a-424c-b935-2458e4c43373 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:45.026012 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d12597e-209a-424c-b935-2458e4c43373 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:45.030921 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6153c-c51a-4900-9563-c15bc06823ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:45.031540 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6153c-c51a-4900-9563-c15bc06823ba admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:45.032640 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6153c-c51a-4900-9563-c15bc06823ba admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:45.032917 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6153c-c51a-4900-9563-c15bc06823ba admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:45.033631 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6153c-c51a-4900-9563-c15bc06823ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:45.035241 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6153c-c51a-4900-9563-c15bc06823ba admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:45.047993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 833/1629] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:45.052773 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6153c-c51a-4900-9563-c15bc06823ba admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:45.054440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 797/1630] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:43:44 2022] PUT /identity/v3/projects/07f5bd35d90f425280b542cdeae03139/users/16825cf32f494c6b97776ff1ba38ec4b/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:45.055896 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb75f5-2072-48e5-a333-f2444b1f3b19 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:45.068654 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6153c-c51a-4900-9563-c15bc06823ba admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:45.119096 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:45.150527 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb9e97-52a4-43ec-853e-48d4a9477438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:45.151211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb9e97-52a4-43ec-853e-48d4a9477438 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:45.151486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb9e97-52a4-43ec-853e-48d4a9477438 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:45.151757 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb9e97-52a4-43ec-853e-48d4a9477438 admin admin] PATH_INFO: `/v3/projects/4fa71891cd6f41a89cb770b5d1c47b8b/users/1d0ef9839e2a49ca9221640b2f9588ef/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:45.153756 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb9e97-52a4-43ec-853e-48d4a9477438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa71891cd6f41a89cb770b5d1c47b8b, user_id=1d0ef9839e2a49ca9221640b2f9588ef, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:45.156563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb9e97-52a4-43ec-853e-48d4a9477438 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:45.182724 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 834/1631] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:43:45 2022] PUT /identity/v3/projects/4fa71891cd6f41a89cb770b5d1c47b8b/users/1d0ef9839e2a49ca9221640b2f9588ef/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:45.182724 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb9e97-52a4-43ec-853e-48d4a9477438 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:45.200574 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb75f5-2072-48e5-a333-f2444b1f3b19 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:45.229871 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:45.285906 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb9e97-52a4-43ec-853e-48d4a9477438 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:45.291850 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3ac9a-0fa3-40b1-a4b3-e98bc15ccac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:45.292501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3ac9a-0fa3-40b1-a4b3-e98bc15ccac5 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:45.292689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d32bb6-d7d8-42ec-bdfe-3bb1eef0d69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:45.293574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3ac9a-0fa3-40b1-a4b3-e98bc15ccac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:45.293574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3ac9a-0fa3-40b1-a4b3-e98bc15ccac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:45.293797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d32bb6-d7d8-42ec-bdfe-3bb1eef0d69e admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:45.293905 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3ac9a-0fa3-40b1-a4b3-e98bc15ccac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:45.294862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d32bb6-d7d8-42ec-bdfe-3bb1eef0d69e admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:45.294862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d32bb6-d7d8-42ec-bdfe-3bb1eef0d69e admin admin] PATH_INFO: `/v3/projects/9e81731014aa4e0ca69db8ef7872c1d7/users/638ddc8b3ff1480b81fa6c5282ac9a63/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:45.295400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3ac9a-0fa3-40b1-a4b3-e98bc15ccac5 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:45.312712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 835/1632] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:45.312712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e54fb5-3320-485e-ae88-215ac302207d None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:45.312712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e54fb5-3320-485e-ae88-215ac302207d None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:45.312712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e54fb5-3320-485e-ae88-215ac302207d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:45.337399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d32bb6-d7d8-42ec-bdfe-3bb1eef0d69e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e81731014aa4e0ca69db8ef7872c1d7, user_id=638ddc8b3ff1480b81fa6c5282ac9a63, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:45.341163 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d32bb6-d7d8-42ec-bdfe-3bb1eef0d69e admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:45.360833 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-20e54fb5-3320-485e-ae88-215ac302207d None None] MFA Rules not processed for user `1d0ef9839e2a49ca9221640b2f9588ef`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:45.362337 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-20e54fb5-3320-485e-ae88-215ac302207d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:45.466407 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 836/1633] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:45 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:45.468258 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 798/1634] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:43:45 2022] PUT /identity/v3/projects/9e81731014aa4e0ca69db8ef7872c1d7/users/638ddc8b3ff1480b81fa6c5282ac9a63/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:45.468258 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf536c-1013-4a25-b915-a6e4e2ce9344 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:45.468581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf536c-1013-4a25-b915-a6e4e2ce9344 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:45.469015 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf536c-1013-4a25-b915-a6e4e2ce9344 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:45.473763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 799/1635] 10.222.0.59 () {54 vars in 912 bytes} [Mon Aug 29 22:43:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:43:45.474750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf536c-1013-4a25-b915-a6e4e2ce9344 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:45.478701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e54fb5-3320-485e-ae88-215ac302207d None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:45.510509 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e54fb5-3320-485e-ae88-215ac302207d None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:45.643125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf536c-1013-4a25-b915-a6e4e2ce9344 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:45.653896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c889e-a779-4324-ab08-e77af016da65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:45.658165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c889e-a779-4324-ab08-e77af016da65 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:45.658165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c889e-a779-4324-ab08-e77af016da65 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:45.658165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c889e-a779-4324-ab08-e77af016da65 admin admin] PATH_INFO: `/v3/projects/07f5bd35d90f425280b542cdeae03139/users/16825cf32f494c6b97776ff1ba38ec4b/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:45.658165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c889e-a779-4324-ab08-e77af016da65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07f5bd35d90f425280b542cdeae03139, user_id=16825cf32f494c6b97776ff1ba38ec4b, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:45.659908 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f5c25-927f-4421-831d-18ca78353842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:45.660748 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f5c25-927f-4421-831d-18ca78353842 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:45.661037 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f5c25-927f-4421-831d-18ca78353842 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:45.661193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c889e-a779-4324-ab08-e77af016da65 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:45.661310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f5c25-927f-4421-831d-18ca78353842 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:45.921875 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f5c25-927f-4421-831d-18ca78353842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:45.921875 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f5c25-927f-4421-831d-18ca78353842 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:45.925490 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:45.925490 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 837/1636] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:43:45 2022] PUT /identity/v3/projects/07f5bd35d90f425280b542cdeae03139/users/16825cf32f494c6b97776ff1ba38ec4b/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:45.925490 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c889e-a779-4324-ab08-e77af016da65 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:45.927750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 800/1637] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:45.927750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e06dc6-1e68-4af0-ba61-28092de49aa9 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:45.927750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e06dc6-1e68-4af0-ba61-28092de49aa9 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:45.927750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e06dc6-1e68-4af0-ba61-28092de49aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:45.927750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-61e06dc6-1e68-4af0-ba61-28092de49aa9 None None] MFA Rules not processed for user `7f21105679f245a19ea4850e6c010dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:45.927750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-61e06dc6-1e68-4af0-ba61-28092de49aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:45.928685 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:45.929111 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c889e-a779-4324-ab08-e77af016da65 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:45.929111 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff475bb-1756-46a3-9d62-6e4abb851d27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:45.929111 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff475bb-1756-46a3-9d62-6e4abb851d27 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:45.929111 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff475bb-1756-46a3-9d62-6e4abb851d27 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:45.929924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff475bb-1756-46a3-9d62-6e4abb851d27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:45.931250 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 801/1638] 10.222.0.59 () {58 vars in 1009 bytes} [Mon Aug 29 22:43:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:45.931250 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53a2ec-59a0-4f58-b846-ae103662aab5 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:45.933091 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53a2ec-59a0-4f58-b846-ae103662aab5 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:45.933091 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53a2ec-59a0-4f58-b846-ae103662aab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:45.936120 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff475bb-1756-46a3-9d62-6e4abb851d27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:45.938203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff475bb-1756-46a3-9d62-6e4abb851d27 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:45.955125 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 838/1639] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:45 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:43:45.955662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff475bb-1756-46a3-9d62-6e4abb851d27 service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:46.025582 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-4d53a2ec-59a0-4f58-b846-ae103662aab5 None None] MFA Rules not processed for user `16825cf32f494c6b97776ff1ba38ec4b`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:46.026811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-4d53a2ec-59a0-4f58-b846-ae103662aab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:46.032772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff475bb-1756-46a3-9d62-6e4abb851d27 service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:46.041264 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f75649-bb4c-4fad-88e5-576143af8c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:46.041671 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f75649-bb4c-4fad-88e5-576143af8c65 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:46.042048 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f75649-bb4c-4fad-88e5-576143af8c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:46.042941 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f75649-bb4c-4fad-88e5-576143af8c65 admin admin] PATH_INFO: `/v3/projects/9e81731014aa4e0ca69db8ef7872c1d7/users/638ddc8b3ff1480b81fa6c5282ac9a63/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:46.045261 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f75649-bb4c-4fad-88e5-576143af8c65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e81731014aa4e0ca69db8ef7872c1d7, user_id=638ddc8b3ff1480b81fa6c5282ac9a63, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:46.050407 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f75649-bb4c-4fad-88e5-576143af8c65 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:46.072765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 839/1640] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:43:45 2022] PUT /identity/v3/projects/9e81731014aa4e0ca69db8ef7872c1d7/users/638ddc8b3ff1480b81fa6c5282ac9a63/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:46.072765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f75649-bb4c-4fad-88e5-576143af8c65 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:46.165218 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 802/1641] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:45 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:46.169994 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f870775-73c7-4365-9710-ff05ccd49f97 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:46.170492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f870775-73c7-4365-9710-ff05ccd49f97 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:46.171098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f870775-73c7-4365-9710-ff05ccd49f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:46.203281 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f75649-bb4c-4fad-88e5-576143af8c65 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:46.223778 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e298e33-324e-4002-80e0-5a2d0774a7dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:46.224729 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e298e33-324e-4002-80e0-5a2d0774a7dc service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:46.225237 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e298e33-324e-4002-80e0-5a2d0774a7dc service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:46.225685 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e298e33-324e-4002-80e0-5a2d0774a7dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:46.254952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-5f870775-73c7-4365-9710-ff05ccd49f97 None None] MFA Rules not processed for user `638ddc8b3ff1480b81fa6c5282ac9a63`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:46.258202 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-5f870775-73c7-4365-9710-ff05ccd49f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:46.412170 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e298e33-324e-4002-80e0-5a2d0774a7dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:46.416562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e298e33-324e-4002-80e0-5a2d0774a7dc service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:46.463003 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 840/1642] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:43:46.464586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e298e33-324e-4002-80e0-5a2d0774a7dc service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:46.482278 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 803/1643] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:46.486142 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e298e33-324e-4002-80e0-5a2d0774a7dc service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:46.494493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aaefa0-96f2-4992-a177-2454b66207c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:46.496666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aaefa0-96f2-4992-a177-2454b66207c6 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:46.496944 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aaefa0-96f2-4992-a177-2454b66207c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:46.502407 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aaefa0-96f2-4992-a177-2454b66207c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:46.584732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:46.626182 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aaefa0-96f2-4992-a177-2454b66207c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:46.628000 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aaefa0-96f2-4992-a177-2454b66207c6 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:46.642490 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 841/1644] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:43:46.973817 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f870775-73c7-4365-9710-ff05ccd49f97 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:46.988383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f870775-73c7-4365-9710-ff05ccd49f97 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:46.999232 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96170857-b3da-46ff-bdba-9a7f52a1d7b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:47.000024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96170857-b3da-46ff-bdba-9a7f52a1d7b7 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:47.000366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96170857-b3da-46ff-bdba-9a7f52a1d7b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:47.000752 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96170857-b3da-46ff-bdba-9a7f52a1d7b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:47.022309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96170857-b3da-46ff-bdba-9a7f52a1d7b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:47.025076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96170857-b3da-46ff-bdba-9a7f52a1d7b7 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:47.034755 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:47.049560 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 804/1645] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:43:47.293371 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aaefa0-96f2-4992-a177-2454b66207c6 service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:47.307507 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aaefa0-96f2-4992-a177-2454b66207c6 service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:47.319222 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc99b234-0e9c-4063-b6f8-43f504ac68f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:47.325338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99b234-0e9c-4063-b6f8-43f504ac68f0 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:47.325729 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99b234-0e9c-4063-b6f8-43f504ac68f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:47.326075 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc99b234-0e9c-4063-b6f8-43f504ac68f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:47.464335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc99b234-0e9c-4063-b6f8-43f504ac68f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:47.466981 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc99b234-0e9c-4063-b6f8-43f504ac68f0 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:47.479218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 842/1646] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:43:47.815947 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96170857-b3da-46ff-bdba-9a7f52a1d7b7 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:47.827291 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96170857-b3da-46ff-bdba-9a7f52a1d7b7 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:47.842475 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb25c55-38b3-4236-a76e-920354aed465 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:47.843487 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb25c55-38b3-4236-a76e-920354aed465 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:47.843487 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb25c55-38b3-4236-a76e-920354aed465 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:47.843641 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb25c55-38b3-4236-a76e-920354aed465 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:47.976111 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb25c55-38b3-4236-a76e-920354aed465 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:47.978802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb25c55-38b3-4236-a76e-920354aed465 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:48.028404 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 805/1647] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:43:49.876063 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c1e64-f303-44c0-a03f-54a540e54215 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:49.876535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c1e64-f303-44c0-a03f-54a540e54215 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:49.876535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c1e64-f303-44c0-a03f-54a540e54215 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:49.879143 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 843/1648] 10.222.0.59 () {54 vars in 962 bytes} [Mon Aug 29 22:43:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:43:49.895314 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ec56f-38b0-4fc8-800f-ac7ae05c96f4 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:49.895314 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ec56f-38b0-4fc8-800f-ac7ae05c96f4 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:49.895314 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ec56f-38b0-4fc8-800f-ac7ae05c96f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:49.942972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-947ec56f-38b0-4fc8-800f-ac7ae05c96f4 None None] MFA Rules not processed for user `7f21105679f245a19ea4850e6c010dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:49.946595 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-947ec56f-38b0-4fc8-800f-ac7ae05c96f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:49.984252 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 806/1649] 10.222.0.59 () {58 vars in 1059 bytes} [Mon Aug 29 22:43:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:49.991129 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b0c0f-fba3-41b3-8b37-0ab69a94b693 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:49.991416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b0c0f-fba3-41b3-8b37-0ab69a94b693 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:49.991782 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b0c0f-fba3-41b3-8b37-0ab69a94b693 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:49.994260 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 844/1650] 10.222.0.59 () {54 vars in 912 bytes} [Mon Aug 29 22:43:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:43:50.006608 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668f649-2e86-4d2f-ab66-6e8366222a34 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:50.006865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668f649-2e86-4d2f-ab66-6e8366222a34 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:50.007228 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668f649-2e86-4d2f-ab66-6e8366222a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:50.064845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-1668f649-2e86-4d2f-ab66-6e8366222a34 None None] MFA Rules not processed for user `7f21105679f245a19ea4850e6c010dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:50.066372 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-1668f649-2e86-4d2f-ab66-6e8366222a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:50.086781 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 807/1651] 10.222.0.59 () {58 vars in 1009 bytes} [Mon Aug 29 22:43:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:50.233702 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b0c0f-fba3-41b3-8b37-0ab69a94b693 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:50.260415 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b0c0f-fba3-41b3-8b37-0ab69a94b693 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:50.266273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacebfe-24a5-4dfc-aa11-daeea38612ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:50.266997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacebfe-24a5-4dfc-aa11-daeea38612ff service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:50.269601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacebfe-24a5-4dfc-aa11-daeea38612ff service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:50.269601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacebfe-24a5-4dfc-aa11-daeea38612ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:50.294119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dacebfe-24a5-4dfc-aa11-daeea38612ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:50.298311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dacebfe-24a5-4dfc-aa11-daeea38612ff service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:50.336970 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 845/1652] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:43:50.478233 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668f649-2e86-4d2f-ab66-6e8366222a34 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:50.587556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668f649-2e86-4d2f-ab66-6e8366222a34 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:50.631194 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883e8bc-39b3-43f9-af66-2a14d5d3eac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:50.632010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883e8bc-39b3-43f9-af66-2a14d5d3eac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:50.632319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883e8bc-39b3-43f9-af66-2a14d5d3eac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:50.632592 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883e8bc-39b3-43f9-af66-2a14d5d3eac1 admin admin] PATH_INFO: `/v3/users/c92e8f3339c44c5bb2a3a385d42d4f68` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:50.634911 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d883e8bc-39b3-43f9-af66-2a14d5d3eac1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c92e8f3339c44c5bb2a3a385d42d4f68)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:50.638391 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d883e8bc-39b3-43f9-af66-2a14d5d3eac1 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:50.942276 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d883e8bc-39b3-43f9-af66-2a14d5d3eac1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c92e8f3339c44c5bb2a3a385d42d4f68'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:50.963813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d883e8bc-39b3-43f9-af66-2a14d5d3eac1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c92e8f3339c44c5bb2a3a385d42d4f68'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:50.971406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d883e8bc-39b3-43f9-af66-2a14d5d3eac1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c92e8f3339c44c5bb2a3a385d42d4f68'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:50.993186 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d883e8bc-39b3-43f9-af66-2a14d5d3eac1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c92e8f3339c44c5bb2a3a385d42d4f68'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:50.994172 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d883e8bc-39b3-43f9-af66-2a14d5d3eac1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c92e8f3339c44c5bb2a3a385d42d4f68'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:51.142373 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 808/1653] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:43:50 2022] DELETE /identity/v3/users/c92e8f3339c44c5bb2a3a385d42d4f68 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:51.147135 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f8952-7dd1-41f4-811b-d4e18482a285 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:51.149348 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f8952-7dd1-41f4-811b-d4e18482a285 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:51.149818 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f8952-7dd1-41f4-811b-d4e18482a285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:51.171991 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:51.207391 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-ab3f8952-7dd1-41f4-811b-d4e18482a285 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:51.209151 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-ab3f8952-7dd1-41f4-811b-d4e18482a285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:51.348828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 846/1654] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:51.457175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883e8bc-39b3-43f9-af66-2a14d5d3eac1 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:51.458827 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d883e8bc-39b3-43f9-af66-2a14d5d3eac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:51.467027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:51.667331 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883e8bc-39b3-43f9-af66-2a14d5d3eac1 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:51.687846 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb5af3-a195-44c9-ad90-baf4e721ff48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:51.691434 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb5af3-a195-44c9-ad90-baf4e721ff48 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:51.706148 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb5af3-a195-44c9-ad90-baf4e721ff48 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:51.706461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb5af3-a195-44c9-ad90-baf4e721ff48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:51.764487 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb5af3-a195-44c9-ad90-baf4e721ff48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:51.770379 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb5af3-a195-44c9-ad90-baf4e721ff48 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:51.786172 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 809/1655] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:43:52.950024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f8952-7dd1-41f4-811b-d4e18482a285 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:52.952813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-ab3f8952-7dd1-41f4-811b-d4e18482a285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:52.979772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f8952-7dd1-41f4-811b-d4e18482a285 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:52.993876 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49df48-2b0b-4e7a-8ea8-8386102bf7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:52.994569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49df48-2b0b-4e7a-8ea8-8386102bf7b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:52.995017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49df48-2b0b-4e7a-8ea8-8386102bf7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:52.995287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49df48-2b0b-4e7a-8ea8-8386102bf7b8 admin admin] PATH_INFO: `/v3/users/6223d34100414e49af647ddb4406c3cd` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:52.996534 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49df48-2b0b-4e7a-8ea8-8386102bf7b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6223d34100414e49af647ddb4406c3cd)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:52.997751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: WARNING py.warnings [None req-1e49df48-2b0b-4e7a-8ea8-8386102bf7b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:43:52.997751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: warnings.warn(msg) Aug 29 22:43:52.997751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: Aug 29 22:43:52.999008 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49df48-2b0b-4e7a-8ea8-8386102bf7b8 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:53.078927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96c8c8-7734-4134-bbcc-f3ebb24196e2 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:53.079456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96c8c8-7734-4134-bbcc-f3ebb24196e2 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:53.079456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96c8c8-7734-4134-bbcc-f3ebb24196e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:53.170359 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-4f96c8c8-7734-4134-bbcc-f3ebb24196e2 None None] MFA Rules not processed for user `42a287a54c4040ac828ee221a3f2e8c7`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:53.171858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-4f96c8c8-7734-4134-bbcc-f3ebb24196e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:53.406774 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 810/1656] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:53 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:43:53.425232 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96c8c8-7734-4134-bbcc-f3ebb24196e2 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:53.427233 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-4f96c8c8-7734-4134-bbcc-f3ebb24196e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:53.441564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1e49df48-2b0b-4e7a-8ea8-8386102bf7b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6223d34100414e49af647ddb4406c3cd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:53.444608 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1e49df48-2b0b-4e7a-8ea8-8386102bf7b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6223d34100414e49af647ddb4406c3cd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:53.484611 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1e49df48-2b0b-4e7a-8ea8-8386102bf7b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6223d34100414e49af647ddb4406c3cd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:53.564780 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96c8c8-7734-4134-bbcc-f3ebb24196e2 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:53.574353 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10706d9f-7639-42f3-b798-c60a6f7b3866 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:53.575804 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10706d9f-7639-42f3-b798-c60a6f7b3866 service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:53.576249 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10706d9f-7639-42f3-b798-c60a6f7b3866 service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:53.576526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10706d9f-7639-42f3-b798-c60a6f7b3866 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:53.578762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-10706d9f-7639-42f3-b798-c60a6f7b3866 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:53.598463 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10706d9f-7639-42f3-b798-c60a6f7b3866 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:53.601626 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10706d9f-7639-42f3-b798-c60a6f7b3866 service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:53.607606 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1e49df48-2b0b-4e7a-8ea8-8386102bf7b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6223d34100414e49af647ddb4406c3cd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:53.611202 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1e49df48-2b0b-4e7a-8ea8-8386102bf7b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6223d34100414e49af647ddb4406c3cd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:53.618281 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 811/1657] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:53 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:43:53.619275 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10706d9f-7639-42f3-b798-c60a6f7b3866 service nova] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:53.620960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-10706d9f-7639-42f3-b798-c60a6f7b3866 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:53.651665 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10706d9f-7639-42f3-b798-c60a6f7b3866 service nova] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:53.661759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7d849-1852-4070-a811-eb4b5a425a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:53.665977 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7d849-1852-4070-a811-eb4b5a425a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:53.666260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7d849-1852-4070-a811-eb4b5a425a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:53.666530 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7d849-1852-4070-a811-eb4b5a425a1c admin admin] PATH_INFO: `/v3/projects/f27cc7902bb04528b9b9b899bcd25206` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:53.671063 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7d849-1852-4070-a811-eb4b5a425a1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f27cc7902bb04528b9b9b899bcd25206)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:53.673391 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: WARNING py.warnings [None req-dcb7d849-1852-4070-a811-eb4b5a425a1c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 22:43:53.673391 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: warnings.warn(msg) Aug 29 22:43:53.673391 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: Aug 29 22:43:53.675841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7d849-1852-4070-a811-eb4b5a425a1c admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:53.706937 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 847/1658] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:43:52 2022] DELETE /identity/v3/users/6223d34100414e49af647ddb4406c3cd => generated 0 bytes in 761 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:53.713495 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fa703-2f4d-42ef-ba23-cfc487854e69 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:53.713773 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fa703-2f4d-42ef-ba23-cfc487854e69 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:53.714185 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fa703-2f4d-42ef-ba23-cfc487854e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:53.744631 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:53.757409 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-9c6fa703-2f4d-42ef-ba23-cfc487854e69 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:53.759336 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-9c6fa703-2f4d-42ef-ba23-cfc487854e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:53.905375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 848/1659] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:53.943037 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-dcb7d849-1852-4070-a811-eb4b5a425a1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f27cc7902bb04528b9b9b899bcd25206', 'request_id': 'req-dcb7d849-1852-4070-a811-eb4b5a425a1c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:53.993119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fa703-2f4d-42ef-ba23-cfc487854e69 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:53.995348 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-9c6fa703-2f4d-42ef-ba23-cfc487854e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:54.050883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:54.080982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-dcb7d849-1852-4070-a811-eb4b5a425a1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f27cc7902bb04528b9b9b899bcd25206', 'request_id': 'req-dcb7d849-1852-4070-a811-eb4b5a425a1c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:54.095175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-dcb7d849-1852-4070-a811-eb4b5a425a1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f27cc7902bb04528b9b9b899bcd25206', 'request_id': 'req-dcb7d849-1852-4070-a811-eb4b5a425a1c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:54.157330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fa703-2f4d-42ef-ba23-cfc487854e69 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:54.157629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-dcb7d849-1852-4070-a811-eb4b5a425a1c admin admin] The token cache is being invalidate because project f27cc7902bb04528b9b9b899bcd25206 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:43:54.157923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-dcb7d849-1852-4070-a811-eb4b5a425a1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:54.158692 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-dcb7d849-1852-4070-a811-eb4b5a425a1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:54.160853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 812/1660] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:43:53 2022] DELETE /identity/v3/projects/f27cc7902bb04528b9b9b899bcd25206 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:54.165682 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7d849-1852-4070-a811-eb4b5a425a1c admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:54.168886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-dcb7d849-1852-4070-a811-eb4b5a425a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:54.242336 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d1a9b-3f32-4a1a-a2da-17107844d124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:54.243125 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d1a9b-3f32-4a1a-a2da-17107844d124 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:54.243505 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d1a9b-3f32-4a1a-a2da-17107844d124 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:54.244628 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d1a9b-3f32-4a1a-a2da-17107844d124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:54.250840 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-4a3d1a9b-3f32-4a1a-a2da-17107844d124 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:54.272760 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7d849-1852-4070-a811-eb4b5a425a1c admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:54.279537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b5e16-b337-4ae9-938d-8e25d2f2d42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:54.280609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b5e16-b337-4ae9-938d-8e25d2f2d42e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:54.280609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b5e16-b337-4ae9-938d-8e25d2f2d42e admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:54.280890 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b5e16-b337-4ae9-938d-8e25d2f2d42e admin admin] PATH_INFO: `/v3/users/16825cf32f494c6b97776ff1ba38ec4b` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:54.282570 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b5e16-b337-4ae9-938d-8e25d2f2d42e admin admin] RBAC: Authorizing `identity:delete_user(user_id=16825cf32f494c6b97776ff1ba38ec4b)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:54.284260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b5e16-b337-4ae9-938d-8e25d2f2d42e admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:54.460713 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d1a9b-3f32-4a1a-a2da-17107844d124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:54.466757 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d1a9b-3f32-4a1a-a2da-17107844d124 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:54.478463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 849/1661] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:43:54.580361 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-851b5e16-b337-4ae9-938d-8e25d2f2d42e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16825cf32f494c6b97776ff1ba38ec4b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:54.602292 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-851b5e16-b337-4ae9-938d-8e25d2f2d42e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16825cf32f494c6b97776ff1ba38ec4b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:54.604602 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-851b5e16-b337-4ae9-938d-8e25d2f2d42e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16825cf32f494c6b97776ff1ba38ec4b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:54.636247 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-851b5e16-b337-4ae9-938d-8e25d2f2d42e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16825cf32f494c6b97776ff1ba38ec4b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:54.640222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-851b5e16-b337-4ae9-938d-8e25d2f2d42e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16825cf32f494c6b97776ff1ba38ec4b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:54.717942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 813/1662] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:43:54 2022] DELETE /identity/v3/users/16825cf32f494c6b97776ff1ba38ec4b => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:55.954868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d1a9b-3f32-4a1a-a2da-17107844d124 service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:55.957325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-4a3d1a9b-3f32-4a1a-a2da-17107844d124 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:55.982447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d1a9b-3f32-4a1a-a2da-17107844d124 service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:56.079174 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b5e16-b337-4ae9-938d-8e25d2f2d42e admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:56.080752 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-851b5e16-b337-4ae9-938d-8e25d2f2d42e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:56.160984 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:56.202269 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af56b83-36d5-460e-8e9e-fc1ea33cdac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:56.202875 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af56b83-36d5-460e-8e9e-fc1ea33cdac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:56.203157 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af56b83-36d5-460e-8e9e-fc1ea33cdac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:56.203427 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af56b83-36d5-460e-8e9e-fc1ea33cdac5 admin admin] PATH_INFO: `/v3/users/1917db952a4047cebbe6d1b232996e6d` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:56.204738 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af56b83-36d5-460e-8e9e-fc1ea33cdac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1917db952a4047cebbe6d1b232996e6d)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:56.207245 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af56b83-36d5-460e-8e9e-fc1ea33cdac5 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:56.271237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b5e16-b337-4ae9-938d-8e25d2f2d42e admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:56.282378 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf309b-30da-4adc-8cc5-7aea756a5648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:56.283044 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf309b-30da-4adc-8cc5-7aea756a5648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:56.283353 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf309b-30da-4adc-8cc5-7aea756a5648 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:56.284224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf309b-30da-4adc-8cc5-7aea756a5648 admin admin] PATH_INFO: `/v3/projects/e221944c64db423caec3b03179adf454` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:56.286555 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf309b-30da-4adc-8cc5-7aea756a5648 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e221944c64db423caec3b03179adf454)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:56.288616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf309b-30da-4adc-8cc5-7aea756a5648 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:56.518103 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-9af56b83-36d5-460e-8e9e-fc1ea33cdac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1917db952a4047cebbe6d1b232996e6d'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:56.522606 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-9af56b83-36d5-460e-8e9e-fc1ea33cdac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1917db952a4047cebbe6d1b232996e6d'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:56.542882 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-9af56b83-36d5-460e-8e9e-fc1ea33cdac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1917db952a4047cebbe6d1b232996e6d'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:56.556247 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d8cf309b-30da-4adc-8cc5-7aea756a5648 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e221944c64db423caec3b03179adf454', 'request_id': 'req-d8cf309b-30da-4adc-8cc5-7aea756a5648'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:56.636161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-9af56b83-36d5-460e-8e9e-fc1ea33cdac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1917db952a4047cebbe6d1b232996e6d'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:56.638978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-9af56b83-36d5-460e-8e9e-fc1ea33cdac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1917db952a4047cebbe6d1b232996e6d'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:56.691999 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 850/1663] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:43:55 2022] DELETE /identity/v3/users/1917db952a4047cebbe6d1b232996e6d => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:56.693547 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af56b83-36d5-460e-8e9e-fc1ea33cdac5 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:56.698460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-9af56b83-36d5-460e-8e9e-fc1ea33cdac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:56.734750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d8cf309b-30da-4adc-8cc5-7aea756a5648 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e221944c64db423caec3b03179adf454', 'request_id': 'req-d8cf309b-30da-4adc-8cc5-7aea756a5648'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:56.748365 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d8cf309b-30da-4adc-8cc5-7aea756a5648 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e221944c64db423caec3b03179adf454', 'request_id': 'req-d8cf309b-30da-4adc-8cc5-7aea756a5648'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:56.782990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d8cf309b-30da-4adc-8cc5-7aea756a5648 admin admin] The token cache is being invalidate because project e221944c64db423caec3b03179adf454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:43:56.783573 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d8cf309b-30da-4adc-8cc5-7aea756a5648 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:56.784730 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d8cf309b-30da-4adc-8cc5-7aea756a5648 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:56.792466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 814/1664] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:43:56 2022] DELETE /identity/v3/projects/e221944c64db423caec3b03179adf454 => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:43:56.795223 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1d31a-b117-4dc9-81a4-66fabf81ab51 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:56.795624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1d31a-b117-4dc9-81a4-66fabf81ab51 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:56.796013 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1d31a-b117-4dc9-81a4-66fabf81ab51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:56.867713 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-24e1d31a-b117-4dc9-81a4-66fabf81ab51 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:56.869434 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-24e1d31a-b117-4dc9-81a4-66fabf81ab51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:56.876066 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:56.878583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af56b83-36d5-460e-8e9e-fc1ea33cdac5 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:56.894868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4d762-d9c3-4192-aaf1-935c6373059e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:56.896120 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4d762-d9c3-4192-aaf1-935c6373059e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:56.900285 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 815/1665] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:56.900285 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1d31a-b117-4dc9-81a4-66fabf81ab51 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:56.900647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4d762-d9c3-4192-aaf1-935c6373059e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:56.900647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4d762-d9c3-4192-aaf1-935c6373059e admin admin] PATH_INFO: `/v3/projects/07f5bd35d90f425280b542cdeae03139` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:56.901878 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-24e1d31a-b117-4dc9-81a4-66fabf81ab51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:56.903238 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e4d762-d9c3-4192-aaf1-935c6373059e admin admin] RBAC: Authorizing `identity:delete_project(project_id=07f5bd35d90f425280b542cdeae03139)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:56.908670 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e4d762-d9c3-4192-aaf1-935c6373059e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:56.931611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1d31a-b117-4dc9-81a4-66fabf81ab51 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:56.945966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a963d-a984-4eae-90fc-37a03aa60a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:56.948821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a963d-a984-4eae-90fc-37a03aa60a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:56.948821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a963d-a984-4eae-90fc-37a03aa60a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:56.948821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a963d-a984-4eae-90fc-37a03aa60a79 admin admin] PATH_INFO: `/v3/users/1d0ef9839e2a49ca9221640b2f9588ef` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:56.949557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a963d-a984-4eae-90fc-37a03aa60a79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d0ef9839e2a49ca9221640b2f9588ef)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:56.953956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a963d-a984-4eae-90fc-37a03aa60a79 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:57.139210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-c7e4d762-d9c3-4192-aaf1-935c6373059e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07f5bd35d90f425280b542cdeae03139', 'request_id': 'req-c7e4d762-d9c3-4192-aaf1-935c6373059e'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:57.159604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-c7e4d762-d9c3-4192-aaf1-935c6373059e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07f5bd35d90f425280b542cdeae03139', 'request_id': 'req-c7e4d762-d9c3-4192-aaf1-935c6373059e'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:57.179038 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-c7e4d762-d9c3-4192-aaf1-935c6373059e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07f5bd35d90f425280b542cdeae03139', 'request_id': 'req-c7e4d762-d9c3-4192-aaf1-935c6373059e'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:57.262439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-fd2a963d-a984-4eae-90fc-37a03aa60a79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d0ef9839e2a49ca9221640b2f9588ef'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:57.281876 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-fd2a963d-a984-4eae-90fc-37a03aa60a79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d0ef9839e2a49ca9221640b2f9588ef'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:57.284951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-fd2a963d-a984-4eae-90fc-37a03aa60a79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d0ef9839e2a49ca9221640b2f9588ef'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:57.306418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-c7e4d762-d9c3-4192-aaf1-935c6373059e admin admin] The token cache is being invalidate because project 07f5bd35d90f425280b542cdeae03139 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:43:57.306992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-c7e4d762-d9c3-4192-aaf1-935c6373059e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:57.308892 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-c7e4d762-d9c3-4192-aaf1-935c6373059e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:57.323755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 851/1666] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:43:56 2022] DELETE /identity/v3/projects/07f5bd35d90f425280b542cdeae03139 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:57.323942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-fd2a963d-a984-4eae-90fc-37a03aa60a79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d0ef9839e2a49ca9221640b2f9588ef'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:57.324033 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4d762-d9c3-4192-aaf1-935c6373059e admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:57.325919 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-fd2a963d-a984-4eae-90fc-37a03aa60a79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d0ef9839e2a49ca9221640b2f9588ef'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:57.327064 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c7e4d762-d9c3-4192-aaf1-935c6373059e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:57.442299 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 816/1667] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:43:56 2022] DELETE /identity/v3/users/1d0ef9839e2a49ca9221640b2f9588ef => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:57.444086 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a00ef-989d-4edd-b976-ba48bc076cbb None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:57.444653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a00ef-989d-4edd-b976-ba48bc076cbb None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:57.445666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a00ef-989d-4edd-b976-ba48bc076cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:57.506301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-b71a00ef-989d-4edd-b976-ba48bc076cbb None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:57.506761 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4d762-d9c3-4192-aaf1-935c6373059e admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:57.510079 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b71a00ef-989d-4edd-b976-ba48bc076cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:57.662993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdca1aa-43a1-4836-8e01-6ce5c26c522d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:57.664473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdca1aa-43a1-4836-8e01-6ce5c26c522d service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:57.665060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdca1aa-43a1-4836-8e01-6ce5c26c522d service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:57.665491 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdca1aa-43a1-4836-8e01-6ce5c26c522d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:57.667527 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-5cdca1aa-43a1-4836-8e01-6ce5c26c522d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:57.712095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 817/1668] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:57.722038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f235e-7471-4267-bf6c-ce766ece105c None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:57.722493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f235e-7471-4267-bf6c-ce766ece105c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:57.722872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f235e-7471-4267-bf6c-ce766ece105c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:57.765338 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-913f235e-7471-4267-bf6c-ce766ece105c None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:57.767034 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-913f235e-7471-4267-bf6c-ce766ece105c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:57.787849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdca1aa-43a1-4836-8e01-6ce5c26c522d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:57.790394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdca1aa-43a1-4836-8e01-6ce5c26c522d service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:57.807928 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 852/1669] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:43:57.810756 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 818/1670] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:57.816321 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdca1aa-43a1-4836-8e01-6ce5c26c522d service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:57.832723 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdca1aa-43a1-4836-8e01-6ce5c26c522d service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:57.841971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177a747-cbeb-4b74-8c13-945ea63bca57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:57.842607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177a747-cbeb-4b74-8c13-945ea63bca57 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:57.842889 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177a747-cbeb-4b74-8c13-945ea63bca57 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:57.843163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177a747-cbeb-4b74-8c13-945ea63bca57 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:57.843824 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177a747-cbeb-4b74-8c13-945ea63bca57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:43:57.844128 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177a747-cbeb-4b74-8c13-945ea63bca57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:57.846295 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177a747-cbeb-4b74-8c13-945ea63bca57 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:57.868256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f235e-7471-4267-bf6c-ce766ece105c None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:57.868925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 853/1671] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:43:57.886652 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f235e-7471-4267-bf6c-ce766ece105c None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:57.894512 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7ad9d-6577-4b7f-9514-5ab05c5c47bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:57.894912 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7ad9d-6577-4b7f-9514-5ab05c5c47bf admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:57.895295 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7ad9d-6577-4b7f-9514-5ab05c5c47bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:57.895638 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7ad9d-6577-4b7f-9514-5ab05c5c47bf admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:57.896644 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7ad9d-6577-4b7f-9514-5ab05c5c47bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:57.900622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7ad9d-6577-4b7f-9514-5ab05c5c47bf admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:58.050665 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 819/1672] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:43:57 2022] POST /identity/v3/projects => generated 387 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:43:58.059099 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177a747-cbeb-4b74-8c13-945ea63bca57 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:58.079513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177a747-cbeb-4b74-8c13-945ea63bca57 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:58.254286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdb2cc-0834-418d-b3bb-30b02b2a732a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:58.254943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdb2cc-0834-418d-b3bb-30b02b2a732a admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:58.255184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdb2cc-0834-418d-b3bb-30b02b2a732a admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:58.259181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdb2cc-0834-418d-b3bb-30b02b2a732a admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:58.259181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdb2cc-0834-418d-b3bb-30b02b2a732a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:58.268836 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdb2cc-0834-418d-b3bb-30b02b2a732a admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:58.422362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 854/1673] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:43:58 2022] POST /identity/v3/users => generated 294 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:58.426893 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7ad9d-6577-4b7f-9514-5ab05c5c47bf admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:58.443728 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7ad9d-6577-4b7f-9514-5ab05c5c47bf admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:58.466517 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194a7e9-ae89-45a0-bd1e-31d6addaba92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:58.467649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194a7e9-ae89-45a0-bd1e-31d6addaba92 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:58.469049 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194a7e9-ae89-45a0-bd1e-31d6addaba92 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:58.473106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194a7e9-ae89-45a0-bd1e-31d6addaba92 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:58.474080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194a7e9-ae89-45a0-bd1e-31d6addaba92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:58.477915 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194a7e9-ae89-45a0-bd1e-31d6addaba92 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:58.498364 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130dc0b-c563-462f-b93d-9f1672a9cb2e None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:58.502015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130dc0b-c563-462f-b93d-9f1672a9cb2e None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:58.502015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130dc0b-c563-462f-b93d-9f1672a9cb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:58.503958 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:58.517372 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 820/1674] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:43:58.522331 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194a7e9-ae89-45a0-bd1e-31d6addaba92 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:58.547596 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194a7e9-ae89-45a0-bd1e-31d6addaba92 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:58.580886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374a56b-3319-452b-b777-a85f5715af02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:58.582401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374a56b-3319-452b-b777-a85f5715af02 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:58.583134 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374a56b-3319-452b-b777-a85f5715af02 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:58.583641 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374a56b-3319-452b-b777-a85f5715af02 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:58.584645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374a56b-3319-452b-b777-a85f5715af02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:58.592829 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374a56b-3319-452b-b777-a85f5715af02 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:58.611077 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-e130dc0b-c563-462f-b93d-9f1672a9cb2e None None] MFA Rules not processed for user `43cdaa09925c4b00be3b45cea4fd40b7`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:58.612319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e130dc0b-c563-462f-b93d-9f1672a9cb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:58.652469 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 821/1675] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:43:58.655379 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374a56b-3319-452b-b777-a85f5715af02 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:58.692287 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374a56b-3319-452b-b777-a85f5715af02 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:58.710926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9cf6c-7134-4afb-82ab-045ea5e5878e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:58.710926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9cf6c-7134-4afb-82ab-045ea5e5878e admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:58.710926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9cf6c-7134-4afb-82ab-045ea5e5878e admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:58.710926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9cf6c-7134-4afb-82ab-045ea5e5878e admin admin] PATH_INFO: `/v3/projects/c032e85afc1e448989f8fc7086480bf4/users/ef7ac604607f40bf882d26fcd9280b3a/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:58.772974 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:43:58.793744 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9cf6c-7134-4afb-82ab-045ea5e5878e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c032e85afc1e448989f8fc7086480bf4, user_id=ef7ac604607f40bf882d26fcd9280b3a, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:58.796942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9cf6c-7134-4afb-82ab-045ea5e5878e admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:58.844250 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 855/1676] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:58.845738 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130dc0b-c563-462f-b93d-9f1672a9cb2e None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:58.847241 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e130dc0b-c563-462f-b93d-9f1672a9cb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:58.916915 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130dc0b-c563-462f-b93d-9f1672a9cb2e None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:58.928662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fcf63-90b4-4fe9-9c90-fa33b02affb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:58.929355 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fcf63-90b4-4fe9-9c90-fa33b02affb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:58.929635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fcf63-90b4-4fe9-9c90-fa33b02affb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:58.930897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fcf63-90b4-4fe9-9c90-fa33b02affb0 admin admin] PATH_INFO: `/v3/users/75eb21cb023f482ca5cba476a315a32f` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:58.932520 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1fcf63-90b4-4fe9-9c90-fa33b02affb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75eb21cb023f482ca5cba476a315a32f)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:58.934529 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1fcf63-90b4-4fe9-9c90-fa33b02affb0 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:58.935760 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 822/1677] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:43:58 2022] PUT /identity/v3/projects/c032e85afc1e448989f8fc7086480bf4/users/ef7ac604607f40bf882d26fcd9280b3a/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:58.936809 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9cf6c-7134-4afb-82ab-045ea5e5878e admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:58.948226 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d4b9cf6c-7134-4afb-82ab-045ea5e5878e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:59.154635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9cf6c-7134-4afb-82ab-045ea5e5878e admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:59.163262 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd7239-c7b7-4328-b5bc-fd901e2f92dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:59.166333 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd7239-c7b7-4328-b5bc-fd901e2f92dc service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:59.166652 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd7239-c7b7-4328-b5bc-fd901e2f92dc service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:59.166897 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd7239-c7b7-4328-b5bc-fd901e2f92dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:59.295779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ba1fcf63-90b4-4fe9-9c90-fa33b02affb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75eb21cb023f482ca5cba476a315a32f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:59.296996 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ba1fcf63-90b4-4fe9-9c90-fa33b02affb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75eb21cb023f482ca5cba476a315a32f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:59.318308 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ba1fcf63-90b4-4fe9-9c90-fa33b02affb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75eb21cb023f482ca5cba476a315a32f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:59.320318 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd7239-c7b7-4328-b5bc-fd901e2f92dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:59.337519 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd7239-c7b7-4328-b5bc-fd901e2f92dc service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:59.359283 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 823/1678] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:43:59.360239 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215e99b-c331-451b-b738-2b5bfe4e0d0b None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:59.360585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215e99b-c331-451b-b738-2b5bfe4e0d0b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:59.360963 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215e99b-c331-451b-b738-2b5bfe4e0d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:59.416576 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ba1fcf63-90b4-4fe9-9c90-fa33b02affb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75eb21cb023f482ca5cba476a315a32f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:59.416790 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-d215e99b-c331-451b-b738-2b5bfe4e0d0b None None] MFA Rules not processed for user `ef7ac604607f40bf882d26fcd9280b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:59.420695 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ba1fcf63-90b4-4fe9-9c90-fa33b02affb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75eb21cb023f482ca5cba476a315a32f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:43:59.428642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d215e99b-c331-451b-b738-2b5bfe4e0d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:59.483772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 856/1679] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:43:58 2022] DELETE /identity/v3/users/75eb21cb023f482ca5cba476a315a32f => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:43:59.485021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fcf63-90b4-4fe9-9c90-fa33b02affb0 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:59.494606 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-ba1fcf63-90b4-4fe9-9c90-fa33b02affb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:59.595221 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 824/1680] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:59.604828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512195f4-42e1-4f99-9dbf-218aee1834b0 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:59.605126 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512195f4-42e1-4f99-9dbf-218aee1834b0 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:59.605406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512195f4-42e1-4f99-9dbf-218aee1834b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:59.702212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-512195f4-42e1-4f99-9dbf-218aee1834b0 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:43:59.703493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-512195f4-42e1-4f99-9dbf-218aee1834b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:59.716652 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 825/1681] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:43:59.721150 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512195f4-42e1-4f99-9dbf-218aee1834b0 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:59.722964 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-512195f4-42e1-4f99-9dbf-218aee1834b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:43:59.725883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fcf63-90b4-4fe9-9c90-fa33b02affb0 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:59.754708 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512195f4-42e1-4f99-9dbf-218aee1834b0 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:43:59.759915 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0ff3f-c102-4158-a982-0bac9d0203d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:59.760549 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0ff3f-c102-4158-a982-0bac9d0203d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:59.761714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0ff3f-c102-4158-a982-0bac9d0203d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:59.766790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0ff3f-c102-4158-a982-0bac9d0203d8 admin admin] PATH_INFO: `/v3/projects/63c73254461e455e9a3971730dc6e0d5` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:59.768355 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0ff3f-c102-4158-a982-0bac9d0203d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63c73254461e455e9a3971730dc6e0d5)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:59.780356 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0ff3f-c102-4158-a982-0bac9d0203d8 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:59.780491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4abf2f-e8c8-44d4-9e8e-91c2e5907759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:43:59.781378 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4abf2f-e8c8-44d4-9e8e-91c2e5907759 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:43:59.781903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4abf2f-e8c8-44d4-9e8e-91c2e5907759 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:43:59.782275 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4abf2f-e8c8-44d4-9e8e-91c2e5907759 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:43:59.783220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4abf2f-e8c8-44d4-9e8e-91c2e5907759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:43:59.785382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4abf2f-e8c8-44d4-9e8e-91c2e5907759 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:43:59.958471 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 826/1682] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:43:59 2022] POST /identity/v3/projects => generated 385 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:43:59.959913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4abf2f-e8c8-44d4-9e8e-91c2e5907759 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:43:59.971307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-5e4abf2f-e8c8-44d4-9e8e-91c2e5907759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:00.013239 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-69b0ff3f-c102-4158-a982-0bac9d0203d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63c73254461e455e9a3971730dc6e0d5', 'request_id': 'req-69b0ff3f-c102-4158-a982-0bac9d0203d8'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:00.037077 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-69b0ff3f-c102-4158-a982-0bac9d0203d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63c73254461e455e9a3971730dc6e0d5', 'request_id': 'req-69b0ff3f-c102-4158-a982-0bac9d0203d8'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:00.064335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-69b0ff3f-c102-4158-a982-0bac9d0203d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63c73254461e455e9a3971730dc6e0d5', 'request_id': 'req-69b0ff3f-c102-4158-a982-0bac9d0203d8'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:00.128690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4abf2f-e8c8-44d4-9e8e-91c2e5907759 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:00.202233 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-69b0ff3f-c102-4158-a982-0bac9d0203d8 admin admin] The token cache is being invalidate because project 63c73254461e455e9a3971730dc6e0d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:44:00.202771 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-69b0ff3f-c102-4158-a982-0bac9d0203d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:00.203442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-69b0ff3f-c102-4158-a982-0bac9d0203d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:00.206278 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 857/1683] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:43:59 2022] DELETE /identity/v3/projects/63c73254461e455e9a3971730dc6e0d5 => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:00.207286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0ff3f-c102-4158-a982-0bac9d0203d8 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:00.211841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-69b0ff3f-c102-4158-a982-0bac9d0203d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:00.230136 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ee67c-d376-4e8e-8c46-3db5e39f93e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:00.231255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee67c-d376-4e8e-8c46-3db5e39f93e3 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:00.231470 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee67c-d376-4e8e-8c46-3db5e39f93e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:00.232080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee67c-d376-4e8e-8c46-3db5e39f93e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:00.237202 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-863ee67c-d376-4e8e-8c46-3db5e39f93e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:00.317519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:00.375540 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ee67c-d376-4e8e-8c46-3db5e39f93e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:00.377288 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ee67c-d376-4e8e-8c46-3db5e39f93e3 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:00.384863 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0ff3f-c102-4158-a982-0bac9d0203d8 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:00.392584 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 827/1684] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:43:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:00.400158 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d078b-d2c8-4cd1-8d49-0d599d3e6c59 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:00.400693 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d078b-d2c8-4cd1-8d49-0d599d3e6c59 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:00.400988 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d078b-d2c8-4cd1-8d49-0d599d3e6c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:00.407012 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:00.421604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998ad32-3188-4db0-9777-03301b204e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:00.429171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998ad32-3188-4db0-9777-03301b204e4e admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:00.429171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998ad32-3188-4db0-9777-03301b204e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:00.429171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998ad32-3188-4db0-9777-03301b204e4e admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:00.429171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998ad32-3188-4db0-9777-03301b204e4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:00.429171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998ad32-3188-4db0-9777-03301b204e4e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:00.491254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-719d078b-d2c8-4cd1-8d49-0d599d3e6c59 None None] MFA Rules not processed for user `cc2f5c40aedf47dcb6374d64fcb04add`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:00.494784 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-719d078b-d2c8-4cd1-8d49-0d599d3e6c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:00.584921 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 858/1685] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:00 2022] POST /identity/v3/users => generated 299 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:00.586855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998ad32-3188-4db0-9777-03301b204e4e admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:00.591274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c998ad32-3188-4db0-9777-03301b204e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:00.625833 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998ad32-3188-4db0-9777-03301b204e4e admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:00.640781 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee0bee-4256-4ad3-ae15-fe5792aa938c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:00.642323 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee0bee-4256-4ad3-ae15-fe5792aa938c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:00.642744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee0bee-4256-4ad3-ae15-fe5792aa938c admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:00.645593 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee0bee-4256-4ad3-ae15-fe5792aa938c admin admin] PATH_INFO: `/v3/users/638ddc8b3ff1480b81fa6c5282ac9a63` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:00.645593 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee0bee-4256-4ad3-ae15-fe5792aa938c admin admin] RBAC: Authorizing `identity:delete_user(user_id=638ddc8b3ff1480b81fa6c5282ac9a63)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:00.659160 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 828/1686] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:00.659638 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee0bee-4256-4ad3-ae15-fe5792aa938c admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:00.661949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d078b-d2c8-4cd1-8d49-0d599d3e6c59 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:00.664704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-719d078b-d2c8-4cd1-8d49-0d599d3e6c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:00.710914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d078b-d2c8-4cd1-8d49-0d599d3e6c59 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:00.724956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48bc9aa-1ad4-4461-b873-89240a8f4e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:00.725631 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bc9aa-1ad4-4461-b873-89240a8f4e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:00.725936 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bc9aa-1ad4-4461-b873-89240a8f4e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:00.726210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bc9aa-1ad4-4461-b873-89240a8f4e2a admin admin] PATH_INFO: `/v3/projects/4fa71891cd6f41a89cb770b5d1c47b8b` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:00.729401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48bc9aa-1ad4-4461-b873-89240a8f4e2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fa71891cd6f41a89cb770b5d1c47b8b)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:00.733130 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48bc9aa-1ad4-4461-b873-89240a8f4e2a admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:01.017987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-94ee0bee-4256-4ad3-ae15-fe5792aa938c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '638ddc8b3ff1480b81fa6c5282ac9a63'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:01.018842 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-94ee0bee-4256-4ad3-ae15-fe5792aa938c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '638ddc8b3ff1480b81fa6c5282ac9a63'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:01.020086 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-a48bc9aa-1ad4-4461-b873-89240a8f4e2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fa71891cd6f41a89cb770b5d1c47b8b', 'request_id': 'req-a48bc9aa-1ad4-4461-b873-89240a8f4e2a'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:01.069281 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-94ee0bee-4256-4ad3-ae15-fe5792aa938c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '638ddc8b3ff1480b81fa6c5282ac9a63'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:01.103905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-a48bc9aa-1ad4-4461-b873-89240a8f4e2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fa71891cd6f41a89cb770b5d1c47b8b', 'request_id': 'req-a48bc9aa-1ad4-4461-b873-89240a8f4e2a'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:01.112724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-a48bc9aa-1ad4-4461-b873-89240a8f4e2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fa71891cd6f41a89cb770b5d1c47b8b', 'request_id': 'req-a48bc9aa-1ad4-4461-b873-89240a8f4e2a'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:01.138156 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-a48bc9aa-1ad4-4461-b873-89240a8f4e2a admin admin] The token cache is being invalidate because project 4fa71891cd6f41a89cb770b5d1c47b8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:44:01.138679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-a48bc9aa-1ad4-4461-b873-89240a8f4e2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:01.140506 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-a48bc9aa-1ad4-4461-b873-89240a8f4e2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:01.150788 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 829/1687] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:44:00 2022] DELETE /identity/v3/projects/4fa71891cd6f41a89cb770b5d1c47b8b => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:01.152878 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ce363-fd3d-40b3-af37-081f2efa71f0 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:01.153352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ce363-fd3d-40b3-af37-081f2efa71f0 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:01.153804 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ce363-fd3d-40b3-af37-081f2efa71f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:01.191264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-865ce363-fd3d-40b3-af37-081f2efa71f0 None None] MFA Rules not processed for user `b21dc8a79bce4b26a690d8990c52116c`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:01.193527 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-865ce363-fd3d-40b3-af37-081f2efa71f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:01.206413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-94ee0bee-4256-4ad3-ae15-fe5792aa938c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '638ddc8b3ff1480b81fa6c5282ac9a63'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:01.208623 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-94ee0bee-4256-4ad3-ae15-fe5792aa938c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '638ddc8b3ff1480b81fa6c5282ac9a63'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:01.272523 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 859/1688] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:44:00 2022] DELETE /identity/v3/users/638ddc8b3ff1480b81fa6c5282ac9a63 => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:01.276582 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee0bee-4256-4ad3-ae15-fe5792aa938c admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:01.280491 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-94ee0bee-4256-4ad3-ae15-fe5792aa938c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:01.360940 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 830/1689] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:01.362610 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e308bb7-9e61-4cec-8f1a-e9a43990a33d None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:01.362995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e308bb7-9e61-4cec-8f1a-e9a43990a33d None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:01.366702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e308bb7-9e61-4cec-8f1a-e9a43990a33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:01.401399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-4e308bb7-9e61-4cec-8f1a-e9a43990a33d None None] MFA Rules not processed for user `cc2f5c40aedf47dcb6374d64fcb04add`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:01.405316 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-4e308bb7-9e61-4cec-8f1a-e9a43990a33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:01.491146 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee0bee-4256-4ad3-ae15-fe5792aa938c admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:01.504325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb8211-5b61-4543-8d72-c7e5819b7268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:01.505472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb8211-5b61-4543-8d72-c7e5819b7268 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:01.506591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb8211-5b61-4543-8d72-c7e5819b7268 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:01.508486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb8211-5b61-4543-8d72-c7e5819b7268 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:01.510736 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb8211-5b61-4543-8d72-c7e5819b7268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:01.512685 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 831/1690] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:01.514469 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb8211-5b61-4543-8d72-c7e5819b7268 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:01.516236 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c26ecd-cc39-43c6-bdf4-cb398a036b7d None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:01.516594 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c26ecd-cc39-43c6-bdf4-cb398a036b7d None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:01.516964 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c26ecd-cc39-43c6-bdf4-cb398a036b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:01.538041 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 860/1691] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:01.547349 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67084ba-af55-4948-b642-04ccc422077e None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:01.552343 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67084ba-af55-4948-b642-04ccc422077e None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:01.553317 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67084ba-af55-4948-b642-04ccc422077e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:01.570690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-07c26ecd-cc39-43c6-bdf4-cb398a036b7d None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:01.578744 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-07c26ecd-cc39-43c6-bdf4-cb398a036b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:01.592995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 832/1692] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:01.595875 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-a67084ba-af55-4948-b642-04ccc422077e None None] MFA Rules not processed for user `42a287a54c4040ac828ee221a3f2e8c7`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:01.597359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a67084ba-af55-4948-b642-04ccc422077e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:01.607795 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3c5e8-204f-45a6-a3b6-23d1613d4c28 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:01.607795 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3c5e8-204f-45a6-a3b6-23d1613d4c28 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:01.608982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3c5e8-204f-45a6-a3b6-23d1613d4c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:01.654069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-6ef3c5e8-204f-45a6-a3b6-23d1613d4c28 None None] MFA Rules not processed for user `b21dc8a79bce4b26a690d8990c52116c`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:01.657616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6ef3c5e8-204f-45a6-a3b6-23d1613d4c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:01.708171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 861/1693] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:01.709449 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67084ba-af55-4948-b642-04ccc422077e None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:01.712287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a67084ba-af55-4948-b642-04ccc422077e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:01.786478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 833/1694] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:01.786478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3c5e8-204f-45a6-a3b6-23d1613d4c28 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:01.798080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6ef3c5e8-204f-45a6-a3b6-23d1613d4c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:01.837138 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3c5e8-204f-45a6-a3b6-23d1613d4c28 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:01.840888 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:01.848847 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30b24b-e70e-4390-9e79-ec6bd3be86ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:01.848847 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30b24b-e70e-4390-9e79-ec6bd3be86ba admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:01.848847 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30b24b-e70e-4390-9e79-ec6bd3be86ba admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:01.848847 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30b24b-e70e-4390-9e79-ec6bd3be86ba admin admin] PATH_INFO: `/v3/projects/1ec1b40c800247c3879b9feadf7f3925/users/3c3c876526f2493c857bdeaafe581eff/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:01.887046 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb30b24b-e70e-4390-9e79-ec6bd3be86ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ec1b40c800247c3879b9feadf7f3925, user_id=3c3c876526f2493c857bdeaafe581eff, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:01.905527 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb30b24b-e70e-4390-9e79-ec6bd3be86ba admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:01.914522 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67084ba-af55-4948-b642-04ccc422077e None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:01.930790 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:01.941598 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d806950-ce7e-4bd7-9d78-bced59978204 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:01.942233 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d806950-ce7e-4bd7-9d78-bced59978204 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:01.942511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d806950-ce7e-4bd7-9d78-bced59978204 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:01.942780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d806950-ce7e-4bd7-9d78-bced59978204 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:01.946748 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8d806950-ce7e-4bd7-9d78-bced59978204 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:01.970402 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d806950-ce7e-4bd7-9d78-bced59978204 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:01.974977 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d806950-ce7e-4bd7-9d78-bced59978204 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:01.997204 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 834/1695] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:01 2022] PUT /identity/v3/projects/1ec1b40c800247c3879b9feadf7f3925/users/3c3c876526f2493c857bdeaafe581eff/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:01.997204 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200d0a5-6b77-4600-8ab1-5636c23c0443 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:01.997204 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200d0a5-6b77-4600-8ab1-5636c23c0443 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:01.997204 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200d0a5-6b77-4600-8ab1-5636c23c0443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:01.999824 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 862/1696] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:01 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:02.012450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d806950-ce7e-4bd7-9d78-bced59978204 service nova] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:02.016730 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8d806950-ce7e-4bd7-9d78-bced59978204 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:02.054635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-b200d0a5-6b77-4600-8ab1-5636c23c0443 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:02.055379 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d806950-ce7e-4bd7-9d78-bced59978204 service nova] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:02.058497 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b200d0a5-6b77-4600-8ab1-5636c23c0443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:02.157511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0e50f-6a84-40ad-ad89-7a4a0441a019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:02.158214 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0e50f-6a84-40ad-ad89-7a4a0441a019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:02.160676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0e50f-6a84-40ad-ad89-7a4a0441a019 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:02.161117 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0e50f-6a84-40ad-ad89-7a4a0441a019 admin admin] PATH_INFO: `/v3/projects/8ba704574496408d84b349f1c44ecc80` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:02.163078 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0e50f-6a84-40ad-ad89-7a4a0441a019 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ba704574496408d84b349f1c44ecc80)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:02.176063 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 835/1697] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:02.183160 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200d0a5-6b77-4600-8ab1-5636c23c0443 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:02.183464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0e50f-6a84-40ad-ad89-7a4a0441a019 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:02.205076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200d0a5-6b77-4600-8ab1-5636c23c0443 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:02.218270 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b48a4-7f68-45b1-a377-7f59529ff1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:02.218908 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b48a4-7f68-45b1-a377-7f59529ff1cf admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:02.219184 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b48a4-7f68-45b1-a377-7f59529ff1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:02.219459 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b48a4-7f68-45b1-a377-7f59529ff1cf admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:02.220103 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b48a4-7f68-45b1-a377-7f59529ff1cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:02.223582 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b48a4-7f68-45b1-a377-7f59529ff1cf admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:02.253655 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 836/1698] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:02.254658 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b48a4-7f68-45b1-a377-7f59529ff1cf admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:02.268450 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b48a4-7f68-45b1-a377-7f59529ff1cf admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:02.274509 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d36944-1021-4f1e-9fee-e6f6a64eb772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:02.275145 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d36944-1021-4f1e-9fee-e6f6a64eb772 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:02.275430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d36944-1021-4f1e-9fee-e6f6a64eb772 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:02.275703 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d36944-1021-4f1e-9fee-e6f6a64eb772 admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:02.276548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d36944-1021-4f1e-9fee-e6f6a64eb772 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:44:02.276860 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d36944-1021-4f1e-9fee-e6f6a64eb772 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:02.278507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d36944-1021-4f1e-9fee-e6f6a64eb772 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:02.297504 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 837/1699] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:44:02.298524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d36944-1021-4f1e-9fee-e6f6a64eb772 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:02.310547 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d36944-1021-4f1e-9fee-e6f6a64eb772 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:02.316463 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16812b-c1c3-4dc5-8bcb-9b16dff6c9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:02.317641 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16812b-c1c3-4dc5-8bcb-9b16dff6c9ec admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:02.317926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16812b-c1c3-4dc5-8bcb-9b16dff6c9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:02.318207 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16812b-c1c3-4dc5-8bcb-9b16dff6c9ec admin admin] PATH_INFO: `/v3/projects/1ec1b40c800247c3879b9feadf7f3925/users/3c3c876526f2493c857bdeaafe581eff/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:02.321505 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16812b-c1c3-4dc5-8bcb-9b16dff6c9ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ec1b40c800247c3879b9feadf7f3925, user_id=3c3c876526f2493c857bdeaafe581eff, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:02.324352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16812b-c1c3-4dc5-8bcb-9b16dff6c9ec admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:02.348836 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 838/1700] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:02 2022] PUT /identity/v3/projects/1ec1b40c800247c3879b9feadf7f3925/users/3c3c876526f2493c857bdeaafe581eff/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:02.350036 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16812b-c1c3-4dc5-8bcb-9b16dff6c9ec admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:02.404391 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-cea0e50f-6a84-40ad-ad89-7a4a0441a019 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ba704574496408d84b349f1c44ecc80', 'request_id': 'req-cea0e50f-6a84-40ad-ad89-7a4a0441a019'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:02.421266 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-cea0e50f-6a84-40ad-ad89-7a4a0441a019 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ba704574496408d84b349f1c44ecc80', 'request_id': 'req-cea0e50f-6a84-40ad-ad89-7a4a0441a019'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:02.435037 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-cea0e50f-6a84-40ad-ad89-7a4a0441a019 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ba704574496408d84b349f1c44ecc80', 'request_id': 'req-cea0e50f-6a84-40ad-ad89-7a4a0441a019'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:02.477347 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16812b-c1c3-4dc5-8bcb-9b16dff6c9ec admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:02.553320 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78530996-c91a-4a74-be2c-4da2d9ddc679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:02.554493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78530996-c91a-4a74-be2c-4da2d9ddc679 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:02.554764 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78530996-c91a-4a74-be2c-4da2d9ddc679 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:02.555375 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78530996-c91a-4a74-be2c-4da2d9ddc679 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:02.556030 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78530996-c91a-4a74-be2c-4da2d9ddc679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:02.558930 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78530996-c91a-4a74-be2c-4da2d9ddc679 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:02.580393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-cea0e50f-6a84-40ad-ad89-7a4a0441a019 admin admin] The token cache is being invalidate because project 8ba704574496408d84b349f1c44ecc80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:44:02.584470 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-cea0e50f-6a84-40ad-ad89-7a4a0441a019 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:02.584470 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-cea0e50f-6a84-40ad-ad89-7a4a0441a019 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:02.584470 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 863/1701] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:44:02 2022] DELETE /identity/v3/projects/8ba704574496408d84b349f1c44ecc80 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:02.591827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065952e-d34e-493c-89f4-0be20b4693e9 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:02.592178 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065952e-d34e-493c-89f4-0be20b4693e9 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:02.592439 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065952e-d34e-493c-89f4-0be20b4693e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:02.661596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-9065952e-d34e-493c-89f4-0be20b4693e9 None None] MFA Rules not processed for user `3c3c876526f2493c857bdeaafe581eff`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:02.662822 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-9065952e-d34e-493c-89f4-0be20b4693e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:02.761788 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 864/1702] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:02 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:02.763690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a49607-b555-4966-8b56-ecb58b5aef30 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:02.763922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a49607-b555-4966-8b56-ecb58b5aef30 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:02.764953 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a49607-b555-4966-8b56-ecb58b5aef30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:02.784306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 839/1703] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:44:02 2022] POST /identity/v3/projects => generated 407 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:02.784306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78530996-c91a-4a74-be2c-4da2d9ddc679 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:02.786730 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-c6a49607-b555-4966-8b56-ecb58b5aef30 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:02.787931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c6a49607-b555-4966-8b56-ecb58b5aef30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:02.790045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-78530996-c91a-4a74-be2c-4da2d9ddc679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:02.883051 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78530996-c91a-4a74-be2c-4da2d9ddc679 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:02.888510 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8ddfa-77f9-4b09-888f-a9022bf39c0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:02.889379 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8ddfa-77f9-4b09-888f-a9022bf39c0d service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:02.892274 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8ddfa-77f9-4b09-888f-a9022bf39c0d service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:02.892274 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8ddfa-77f9-4b09-888f-a9022bf39c0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:02.892545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 865/1704] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:02.892545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a49607-b555-4966-8b56-ecb58b5aef30 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:02.893795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c6a49607-b555-4966-8b56-ecb58b5aef30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:02.908698 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a49607-b555-4966-8b56-ecb58b5aef30 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:02.918791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d663163-df6e-4b5e-a16c-d5cc4981a954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:02.921865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d663163-df6e-4b5e-a16c-d5cc4981a954 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:02.922361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d663163-df6e-4b5e-a16c-d5cc4981a954 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:02.926435 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d663163-df6e-4b5e-a16c-d5cc4981a954 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:02.926435 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d663163-df6e-4b5e-a16c-d5cc4981a954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:02.928257 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d663163-df6e-4b5e-a16c-d5cc4981a954 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:02.996295 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8ddfa-77f9-4b09-888f-a9022bf39c0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:02.997893 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8ddfa-77f9-4b09-888f-a9022bf39c0d service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:03.011936 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 840/1705] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:02 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:03.013195 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb24613-5ff3-464b-a113-f06068f1d00e None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:03.013710 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb24613-5ff3-464b-a113-f06068f1d00e None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:03.014062 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb24613-5ff3-464b-a113-f06068f1d00e None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:03.015686 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 841/1706] 10.222.0.59 () {54 vars in 912 bytes} [Mon Aug 29 22:44:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:44:03.021863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb24613-5ff3-464b-a113-f06068f1d00e None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:03.024705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3eb24613-5ff3-464b-a113-f06068f1d00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:03.047006 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 866/1707] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:02 2022] POST /identity/v3/users => generated 304 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:03.048819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a72cc-1793-4065-80fe-5ed0647e6558 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:03.049983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a72cc-1793-4065-80fe-5ed0647e6558 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:03.050588 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a72cc-1793-4065-80fe-5ed0647e6558 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:03.052317 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 867/1708] 10.222.0.59 () {54 vars in 912 bytes} [Mon Aug 29 22:44:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:44:03.056906 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ac149-0c38-40f5-9e06-8849e84b48b6 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:03.057267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ac149-0c38-40f5-9e06-8849e84b48b6 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:03.061691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ac149-0c38-40f5-9e06-8849e84b48b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:03.061827 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb24613-5ff3-464b-a113-f06068f1d00e None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:03.065680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad56a6-381d-45c6-afd9-3edd4f7abdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:03.066703 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad56a6-381d-45c6-afd9-3edd4f7abdb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:03.067124 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad56a6-381d-45c6-afd9-3edd4f7abdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:03.067494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad56a6-381d-45c6-afd9-3edd4f7abdb7 admin admin] PATH_INFO: `/v3/projects/9e81731014aa4e0ca69db8ef7872c1d7` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:03.087693 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad56a6-381d-45c6-afd9-3edd4f7abdb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e81731014aa4e0ca69db8ef7872c1d7)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:03.087693 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad56a6-381d-45c6-afd9-3edd4f7abdb7 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:03.113345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-713ac149-0c38-40f5-9e06-8849e84b48b6 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:03.114631 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-713ac149-0c38-40f5-9e06-8849e84b48b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:03.124260 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 868/1709] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:03.130302 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf64786-d6da-4617-91d5-6efdfa3077a2 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:03.130594 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf64786-d6da-4617-91d5-6efdfa3077a2 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:03.130868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf64786-d6da-4617-91d5-6efdfa3077a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:03.173635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-cdf64786-d6da-4617-91d5-6efdfa3077a2 None None] MFA Rules not processed for user `cb802d01e71b486cb40c6388430d674d`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:03.178131 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-cdf64786-d6da-4617-91d5-6efdfa3077a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:03.237046 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 869/1710] 10.222.0.59 () {58 vars in 1009 bytes} [Mon Aug 29 22:44:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:03.238363 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf64786-d6da-4617-91d5-6efdfa3077a2 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:03.250549 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf64786-d6da-4617-91d5-6efdfa3077a2 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:03.255187 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfa062-8b82-47e8-b65f-b234ec7b23fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:03.255832 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfa062-8b82-47e8-b65f-b234ec7b23fc admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:03.256113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfa062-8b82-47e8-b65f-b234ec7b23fc admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:03.256412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfa062-8b82-47e8-b65f-b234ec7b23fc admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:03.257692 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfa062-8b82-47e8-b65f-b234ec7b23fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:03.259288 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfa062-8b82-47e8-b65f-b234ec7b23fc admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:03.269213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 870/1711] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:03.270510 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa798b-b46d-4524-a53d-0c7f67e654ef None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:03.270798 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa798b-b46d-4524-a53d-0c7f67e654ef None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:03.271074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa798b-b46d-4524-a53d-0c7f67e654ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:03.290682 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-0dfa798b-b46d-4524-a53d-0c7f67e654ef None None] MFA Rules not processed for user `cb802d01e71b486cb40c6388430d674d`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:03.291876 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-0dfa798b-b46d-4524-a53d-0c7f67e654ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:03.298268 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 871/1712] 10.222.0.59 () {58 vars in 1009 bytes} [Mon Aug 29 22:44:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:03.299086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa798b-b46d-4524-a53d-0c7f67e654ef None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:03.315705 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa798b-b46d-4524-a53d-0c7f67e654ef None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:03.325062 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-7fad56a6-381d-45c6-afd9-3edd4f7abdb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e81731014aa4e0ca69db8ef7872c1d7', 'request_id': 'req-7fad56a6-381d-45c6-afd9-3edd4f7abdb7'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:03.404067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f41c8a-c9ed-49e5-a150-5c2557b7b6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:03.406070 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f41c8a-c9ed-49e5-a150-5c2557b7b6a1 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:03.406070 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f41c8a-c9ed-49e5-a150-5c2557b7b6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:03.406285 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f41c8a-c9ed-49e5-a150-5c2557b7b6a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:03.407646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f41c8a-c9ed-49e5-a150-5c2557b7b6a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:44:03.408476 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f41c8a-c9ed-49e5-a150-5c2557b7b6a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:03.411867 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f41c8a-c9ed-49e5-a150-5c2557b7b6a1 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:03.435041 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 872/1713] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:44:03.436247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f41c8a-c9ed-49e5-a150-5c2557b7b6a1 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:03.438638 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-56f41c8a-c9ed-49e5-a150-5c2557b7b6a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:03.447646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-7fad56a6-381d-45c6-afd9-3edd4f7abdb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e81731014aa4e0ca69db8ef7872c1d7', 'request_id': 'req-7fad56a6-381d-45c6-afd9-3edd4f7abdb7'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:03.460742 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-7fad56a6-381d-45c6-afd9-3edd4f7abdb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e81731014aa4e0ca69db8ef7872c1d7', 'request_id': 'req-7fad56a6-381d-45c6-afd9-3edd4f7abdb7'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:03.461558 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f41c8a-c9ed-49e5-a150-5c2557b7b6a1 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:03.471619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da265ae-dec9-4626-93a1-e6d4452d5511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:03.472742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da265ae-dec9-4626-93a1-e6d4452d5511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:03.473393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da265ae-dec9-4626-93a1-e6d4452d5511 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:03.473795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da265ae-dec9-4626-93a1-e6d4452d5511 admin admin] PATH_INFO: `/v3/users/ef7ac604607f40bf882d26fcd9280b3a` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:03.475629 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da265ae-dec9-4626-93a1-e6d4452d5511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef7ac604607f40bf882d26fcd9280b3a)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:03.478672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:03.480032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da265ae-dec9-4626-93a1-e6d4452d5511 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:03.503563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-7fad56a6-381d-45c6-afd9-3edd4f7abdb7 admin admin] The token cache is being invalidate because project 9e81731014aa4e0ca69db8ef7872c1d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:44:03.504120 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-7fad56a6-381d-45c6-afd9-3edd4f7abdb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:03.510248 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-7fad56a6-381d-45c6-afd9-3edd4f7abdb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:03.528287 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 842/1714] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:44:03 2022] DELETE /identity/v3/projects/9e81731014aa4e0ca69db8ef7872c1d7 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:03.530648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad56a6-381d-45c6-afd9-3edd4f7abdb7 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:03.561030 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-7fad56a6-381d-45c6-afd9-3edd4f7abdb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:03.706092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad56a6-381d-45c6-afd9-3edd4f7abdb7 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:03.711271 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4e4c3-6ee3-4dc8-96c4-ac764ece5603 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:03.711944 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4e4c3-6ee3-4dc8-96c4-ac764ece5603 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:03.712322 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4e4c3-6ee3-4dc8-96c4-ac764ece5603 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:03.713113 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4e4c3-6ee3-4dc8-96c4-ac764ece5603 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:03.715312 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-9aa4e4c3-6ee3-4dc8-96c4-ac764ece5603 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:03.726790 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa4e4c3-6ee3-4dc8-96c4-ac764ece5603 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:03.728286 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa4e4c3-6ee3-4dc8-96c4-ac764ece5603 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:03.739114 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 843/1715] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:03.739960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4e4c3-6ee3-4dc8-96c4-ac764ece5603 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:03.741415 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-9aa4e4c3-6ee3-4dc8-96c4-ac764ece5603 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:03.755967 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4e4c3-6ee3-4dc8-96c4-ac764ece5603 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:03.762043 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df90b0-dbfc-4aca-a7e7-4a66dbce4fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:03.762951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df90b0-dbfc-4aca-a7e7-4a66dbce4fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:03.763069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df90b0-dbfc-4aca-a7e7-4a66dbce4fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:03.766889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df90b0-dbfc-4aca-a7e7-4a66dbce4fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:03.766889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df90b0-dbfc-4aca-a7e7-4a66dbce4fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:03.766889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df90b0-dbfc-4aca-a7e7-4a66dbce4fe6 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:03.779075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 844/1716] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:03.783233 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df90b0-dbfc-4aca-a7e7-4a66dbce4fe6 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:03.784755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1da265ae-dec9-4626-93a1-e6d4452d5511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef7ac604607f40bf882d26fcd9280b3a'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:03.785206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-21df90b0-dbfc-4aca-a7e7-4a66dbce4fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:03.786011 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1da265ae-dec9-4626-93a1-e6d4452d5511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef7ac604607f40bf882d26fcd9280b3a'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:03.790752 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=78,keystone:DELETE=17,keystone:INSERT=5 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:03.802218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1da265ae-dec9-4626-93a1-e6d4452d5511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef7ac604607f40bf882d26fcd9280b3a'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:03.802375 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df90b0-dbfc-4aca-a7e7-4a66dbce4fe6 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:03.808432 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6328f-3b84-4a22-bc93-1d01bcc8d398 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:03.809334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6328f-3b84-4a22-bc93-1d01bcc8d398 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:03.810067 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6328f-3b84-4a22-bc93-1d01bcc8d398 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:03.810067 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6328f-3b84-4a22-bc93-1d01bcc8d398 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:03.812433 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a9e6328f-3b84-4a22-bc93-1d01bcc8d398 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:03.825650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6328f-3b84-4a22-bc93-1d01bcc8d398 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:03.827276 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6328f-3b84-4a22-bc93-1d01bcc8d398 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:03.841279 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 845/1717] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:03.842181 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6328f-3b84-4a22-bc93-1d01bcc8d398 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:03.843692 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a9e6328f-3b84-4a22-bc93-1d01bcc8d398 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:03.859571 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6328f-3b84-4a22-bc93-1d01bcc8d398 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:03.865454 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d14ad-7a5e-4821-906c-69cf2a4454af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:03.866142 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d14ad-7a5e-4821-906c-69cf2a4454af admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:03.867093 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d14ad-7a5e-4821-906c-69cf2a4454af admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:03.867545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d14ad-7a5e-4821-906c-69cf2a4454af admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:03.868374 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d14ad-7a5e-4821-906c-69cf2a4454af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:03.870694 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d14ad-7a5e-4821-906c-69cf2a4454af admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:03.903564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1da265ae-dec9-4626-93a1-e6d4452d5511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef7ac604607f40bf882d26fcd9280b3a'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:03.906520 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1da265ae-dec9-4626-93a1-e6d4452d5511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef7ac604607f40bf882d26fcd9280b3a'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:03.969568 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 873/1718] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:44:03 2022] DELETE /identity/v3/users/ef7ac604607f40bf882d26fcd9280b3a => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:03.973772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3794dd-4cf3-40d0-ae06-0b65a0163068 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:03.973772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3794dd-4cf3-40d0-ae06-0b65a0163068 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:03.973772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3794dd-4cf3-40d0-ae06-0b65a0163068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:04.019897 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 846/1719] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:44:03 2022] POST /identity/v3/projects => generated 371 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:04.021264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d14ad-7a5e-4821-906c-69cf2a4454af admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:04.023330 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-1f3d14ad-7a5e-4821-906c-69cf2a4454af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:04.028330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-0e3794dd-4cf3-40d0-ae06-0b65a0163068 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:04.029566 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-0e3794dd-4cf3-40d0-ae06-0b65a0163068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:04.148206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 874/1720] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:04.148206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac4d97-9122-4d91-95ce-6f5a36074094 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:04.148206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac4d97-9122-4d91-95ce-6f5a36074094 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:04.148206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac4d97-9122-4d91-95ce-6f5a36074094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:04.155001 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d14ad-7a5e-4821-906c-69cf2a4454af admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:04.171896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-63ac4d97-9122-4d91-95ce-6f5a36074094 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:04.171896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-63ac4d97-9122-4d91-95ce-6f5a36074094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:04.172194 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906165c1-2a1b-492d-97f5-d10c09feaae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:04.172965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906165c1-2a1b-492d-97f5-d10c09feaae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:04.173351 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906165c1-2a1b-492d-97f5-d10c09feaae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:04.173728 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906165c1-2a1b-492d-97f5-d10c09feaae0 admin admin] PATH_INFO: `/v3/projects/53e6c32266fb4fbe96a125c5d2a81812/users/a40a9d0e8e6d4b15becb654edeeba0b0/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:04.180950 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 875/1721] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:04.185158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac4d97-9122-4d91-95ce-6f5a36074094 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:04.197942 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac4d97-9122-4d91-95ce-6f5a36074094 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:04.204069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb4db5-e3e2-4f46-a4fe-1e4543f34243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:04.206408 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb4db5-e3e2-4f46-a4fe-1e4543f34243 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:04.206408 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb4db5-e3e2-4f46-a4fe-1e4543f34243 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:04.206408 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb4db5-e3e2-4f46-a4fe-1e4543f34243 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:04.206408 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb4db5-e3e2-4f46-a4fe-1e4543f34243 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:04.208241 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb4db5-e3e2-4f46-a4fe-1e4543f34243 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:04.219820 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906165c1-2a1b-492d-97f5-d10c09feaae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53e6c32266fb4fbe96a125c5d2a81812, user_id=a40a9d0e8e6d4b15becb654edeeba0b0, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:04.225142 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906165c1-2a1b-492d-97f5-d10c09feaae0 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:04.341262 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 876/1722] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:04 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:04.345186 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 847/1723] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:04 2022] PUT /identity/v3/projects/53e6c32266fb4fbe96a125c5d2a81812/users/a40a9d0e8e6d4b15becb654edeeba0b0/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:04.347202 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a004a67-6bcd-4da1-b901-709f0ba3db37 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:04.347526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a004a67-6bcd-4da1-b901-709f0ba3db37 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:04.347796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a004a67-6bcd-4da1-b901-709f0ba3db37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:04.350782 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906165c1-2a1b-492d-97f5-d10c09feaae0 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:04.362504 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-906165c1-2a1b-492d-97f5-d10c09feaae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:04.391894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-6a004a67-6bcd-4da1-b901-709f0ba3db37 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:04.393811 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6a004a67-6bcd-4da1-b901-709f0ba3db37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:04.485677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906165c1-2a1b-492d-97f5-d10c09feaae0 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:04.496182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbeca4-6872-4ee9-b523-24229e08d83a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:04.496979 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbeca4-6872-4ee9-b523-24229e08d83a service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:04.497261 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbeca4-6872-4ee9-b523-24229e08d83a service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:04.497579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbeca4-6872-4ee9-b523-24229e08d83a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:04.498018 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 877/1724] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:04.498879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a004a67-6bcd-4da1-b901-709f0ba3db37 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:04.508977 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fbeca4-6872-4ee9-b523-24229e08d83a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:04.510659 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fbeca4-6872-4ee9-b523-24229e08d83a service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:04.510763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a004a67-6bcd-4da1-b901-709f0ba3db37 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:04.516201 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b22c8-43ec-49b6-8d21-0365bd39f06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:04.519808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b22c8-43ec-49b6-8d21-0365bd39f06c admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:04.519808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b22c8-43ec-49b6-8d21-0365bd39f06c admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:04.519808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b22c8-43ec-49b6-8d21-0365bd39f06c admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:04.519808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b22c8-43ec-49b6-8d21-0365bd39f06c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:04.519808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b22c8-43ec-49b6-8d21-0365bd39f06c admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:04.521441 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 848/1725] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:04.523300 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75431ff4-5af1-42d2-adbc-2ef264a54d01 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:04.523583 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75431ff4-5af1-42d2-adbc-2ef264a54d01 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:04.523847 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75431ff4-5af1-42d2-adbc-2ef264a54d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:04.531690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 878/1726] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:04.532402 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b22c8-43ec-49b6-8d21-0365bd39f06c admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:04.559712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b22c8-43ec-49b6-8d21-0365bd39f06c admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:04.561226 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-75431ff4-5af1-42d2-adbc-2ef264a54d01 None None] MFA Rules not processed for user `a40a9d0e8e6d4b15becb654edeeba0b0`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:04.563790 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-75431ff4-5af1-42d2-adbc-2ef264a54d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:04.582368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5feb4-5e03-42fe-abde-8467a2966595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:04.592398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5feb4-5e03-42fe-abde-8467a2966595 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:04.592496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5feb4-5e03-42fe-abde-8467a2966595 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:04.593765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5feb4-5e03-42fe-abde-8467a2966595 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:04.594682 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5feb4-5e03-42fe-abde-8467a2966595 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:44:04.603080 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5feb4-5e03-42fe-abde-8467a2966595 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:04.603080 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5feb4-5e03-42fe-abde-8467a2966595 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:04.623725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 879/1727] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:44:04.624921 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5feb4-5e03-42fe-abde-8467a2966595 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:04.641305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5feb4-5e03-42fe-abde-8467a2966595 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:04.648606 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a29468-9d8d-40cf-b50e-d8cf4692b43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:04.649413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a29468-9d8d-40cf-b50e-d8cf4692b43c admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:04.650810 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a29468-9d8d-40cf-b50e-d8cf4692b43c admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:04.651106 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a29468-9d8d-40cf-b50e-d8cf4692b43c admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:04.651766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a29468-9d8d-40cf-b50e-d8cf4692b43c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:04.653838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a29468-9d8d-40cf-b50e-d8cf4692b43c admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:04.720051 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 880/1728] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:04.720614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 849/1729] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:04 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:04.721914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75431ff4-5af1-42d2-adbc-2ef264a54d01 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:04.723787 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-75431ff4-5af1-42d2-adbc-2ef264a54d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:04.723999 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f57a5-7554-432a-89a7-91f1c8815497 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:04.724378 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f57a5-7554-432a-89a7-91f1c8815497 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:04.724712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f57a5-7554-432a-89a7-91f1c8815497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:04.784329 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-c89f57a5-7554-432a-89a7-91f1c8815497 None None] MFA Rules not processed for user `43cdaa09925c4b00be3b45cea4fd40b7`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:04.787822 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c89f57a5-7554-432a-89a7-91f1c8815497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:04.892006 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75431ff4-5af1-42d2-adbc-2ef264a54d01 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:04.903713 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52176953-02c3-4143-a1c5-29c5cf4dcdc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:04.907043 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52176953-02c3-4143-a1c5-29c5cf4dcdc1 service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:04.907758 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52176953-02c3-4143-a1c5-29c5cf4dcdc1 service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:04.908350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52176953-02c3-4143-a1c5-29c5cf4dcdc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:04.910551 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-52176953-02c3-4143-a1c5-29c5cf4dcdc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:04.912400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 881/1730] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:04.915298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f57a5-7554-432a-89a7-91f1c8815497 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:04.930499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:04.940310 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f57a5-7554-432a-89a7-91f1c8815497 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:04.964511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8cbe1-e250-49a8-bfba-f3c8134fb893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:04.964511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8cbe1-e250-49a8-bfba-f3c8134fb893 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:04.964511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8cbe1-e250-49a8-bfba-f3c8134fb893 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:04.964511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8cbe1-e250-49a8-bfba-f3c8134fb893 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:04.964826 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8cbe1-e250-49a8-bfba-f3c8134fb893 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:04.964826 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8cbe1-e250-49a8-bfba-f3c8134fb893 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:05.054555 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52176953-02c3-4143-a1c5-29c5cf4dcdc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:05.056212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52176953-02c3-4143-a1c5-29c5cf4dcdc1 service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:05.088027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 850/1731] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:05.088420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 882/1732] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:44:04 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:05.089247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8cbe1-e250-49a8-bfba-f3c8134fb893 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:05.089826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52176953-02c3-4143-a1c5-29c5cf4dcdc1 service nova] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:05.119055 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52176953-02c3-4143-a1c5-29c5cf4dcdc1 service nova] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:05.234315 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8cbe1-e250-49a8-bfba-f3c8134fb893 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:05.245349 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d36c6d-072c-4b18-bcea-1e8029f29c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:05.245932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d36c6d-072c-4b18-bcea-1e8029f29c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:05.245932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d36c6d-072c-4b18-bcea-1e8029f29c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:05.246228 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d36c6d-072c-4b18-bcea-1e8029f29c1d admin admin] PATH_INFO: `/v3/projects/ffd51e41f76e4ff0a89de9365049e149/users/5a454af87e5f427487019b237aab286b/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:05.289185 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d36c6d-072c-4b18-bcea-1e8029f29c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffd51e41f76e4ff0a89de9365049e149, user_id=5a454af87e5f427487019b237aab286b, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:05.291945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d36c6d-072c-4b18-bcea-1e8029f29c1d admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:05.302294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71da34e-d257-4fc4-a7a5-6dc75bb3b8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:05.303466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71da34e-d257-4fc4-a7a5-6dc75bb3b8c7 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:05.304381 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71da34e-d257-4fc4-a7a5-6dc75bb3b8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:05.304657 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71da34e-d257-4fc4-a7a5-6dc75bb3b8c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:05.307614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71da34e-d257-4fc4-a7a5-6dc75bb3b8c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:05.310377 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71da34e-d257-4fc4-a7a5-6dc75bb3b8c7 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:05.310833 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:05.378771 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 883/1733] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:05 2022] PUT /identity/v3/projects/ffd51e41f76e4ff0a89de9365049e149/users/5a454af87e5f427487019b237aab286b/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:05.392168 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d36c6d-072c-4b18-bcea-1e8029f29c1d admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:05.396183 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-73d36c6d-072c-4b18-bcea-1e8029f29c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:05.419948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 851/1734] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:44:05 2022] POST /identity/v3/projects => generated 409 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:05.428255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdae2b0-748d-45a3-b0d4-7c5ff0714eec None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:05.428946 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdae2b0-748d-45a3-b0d4-7c5ff0714eec None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:05.429422 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdae2b0-748d-45a3-b0d4-7c5ff0714eec None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:05.431319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 852/1735] 10.222.0.59 () {54 vars in 961 bytes} [Mon Aug 29 22:44:05 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:44:05.446219 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93de48-142c-401c-9a6e-d03941ef328b None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:05.460840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93de48-142c-401c-9a6e-d03941ef328b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:05.464880 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93de48-142c-401c-9a6e-d03941ef328b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:05.514280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-0d93de48-142c-401c-9a6e-d03941ef328b None None] MFA Rules not processed for user `cc2f5c40aedf47dcb6374d64fcb04add`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:05.516499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-0d93de48-142c-401c-9a6e-d03941ef328b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:05.548370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d36c6d-072c-4b18-bcea-1e8029f29c1d admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:05.617895 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:05.672079 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 853/1736] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:05.673350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93de48-142c-401c-9a6e-d03941ef328b None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:05.676414 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:05.699134 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d73b17-e2a7-47c8-96d4-c12241447076 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:05.700996 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d73b17-e2a7-47c8-96d4-c12241447076 service placement] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:05.702640 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d73b17-e2a7-47c8-96d4-c12241447076 service placement] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:05.703038 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d73b17-e2a7-47c8-96d4-c12241447076 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:05.706710 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-82d73b17-e2a7-47c8-96d4-c12241447076 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:05.735882 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d73b17-e2a7-47c8-96d4-c12241447076 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:05.739733 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d73b17-e2a7-47c8-96d4-c12241447076 service placement] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:05.762957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 884/1737] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:05.764553 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fad42d-f8a7-4309-8d31-c5a72477d9c0 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:05.764959 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fad42d-f8a7-4309-8d31-c5a72477d9c0 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:05.765306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fad42d-f8a7-4309-8d31-c5a72477d9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:05.833697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93de48-142c-401c-9a6e-d03941ef328b None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:05.855215 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-00fad42d-f8a7-4309-8d31-c5a72477d9c0 None None] MFA Rules not processed for user `5a454af87e5f427487019b237aab286b`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:05.857851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-00fad42d-f8a7-4309-8d31-c5a72477d9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:05.860853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511c5f3-2c18-46fa-ae23-b19a0a7cad88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:05.861486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511c5f3-2c18-46fa-ae23-b19a0a7cad88 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:05.861948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511c5f3-2c18-46fa-ae23-b19a0a7cad88 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:05.862350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511c5f3-2c18-46fa-ae23-b19a0a7cad88 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:05.863015 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511c5f3-2c18-46fa-ae23-b19a0a7cad88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:05.864981 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511c5f3-2c18-46fa-ae23-b19a0a7cad88 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:06.011876 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 854/1738] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:05 2022] POST /identity/v3/users => generated 292 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:06.013223 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511c5f3-2c18-46fa-ae23-b19a0a7cad88 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:06.014942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d511c5f3-2c18-46fa-ae23-b19a0a7cad88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:06.019901 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 885/1739] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:06.021140 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fad42d-f8a7-4309-8d31-c5a72477d9c0 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:06.048711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511c5f3-2c18-46fa-ae23-b19a0a7cad88 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:06.049332 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fad42d-f8a7-4309-8d31-c5a72477d9c0 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:06.060444 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c77c4-78f9-46d4-860b-365a3af7144e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:06.061372 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c77c4-78f9-46d4-860b-365a3af7144e admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:06.063517 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c77c4-78f9-46d4-860b-365a3af7144e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:06.063517 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c77c4-78f9-46d4-860b-365a3af7144e admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:06.063987 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39add0-057d-4192-b13f-da0aeaac0430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:06.063987 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39add0-057d-4192-b13f-da0aeaac0430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:06.063987 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39add0-057d-4192-b13f-da0aeaac0430 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:06.063987 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39add0-057d-4192-b13f-da0aeaac0430 admin admin] PATH_INFO: `/v3/projects/c032e85afc1e448989f8fc7086480bf4` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:06.064309 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c77c4-78f9-46d4-860b-365a3af7144e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:06.065661 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39add0-057d-4192-b13f-da0aeaac0430 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c032e85afc1e448989f8fc7086480bf4)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:06.066902 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c77c4-78f9-46d4-860b-365a3af7144e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:06.069763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39add0-057d-4192-b13f-da0aeaac0430 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:06.085985 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:06.207571 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 886/1740] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:06 2022] POST /identity/v3/users => generated 311 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:06.208973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0690f6-9283-44a7-84bc-7ea75d3ebfca None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:06.209260 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0690f6-9283-44a7-84bc-7ea75d3ebfca None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:06.210598 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0690f6-9283-44a7-84bc-7ea75d3ebfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:06.249780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-5b0690f6-9283-44a7-84bc-7ea75d3ebfca None None] MFA Rules not processed for user `7215ee41033742d79671f4f42b81cd06`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:06.251065 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-5b0690f6-9283-44a7-84bc-7ea75d3ebfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:06.268655 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1f39add0-057d-4192-b13f-da0aeaac0430 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c032e85afc1e448989f8fc7086480bf4', 'request_id': 'req-1f39add0-057d-4192-b13f-da0aeaac0430'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:06.383217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1f39add0-057d-4192-b13f-da0aeaac0430 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c032e85afc1e448989f8fc7086480bf4', 'request_id': 'req-1f39add0-057d-4192-b13f-da0aeaac0430'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:06.390320 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1f39add0-057d-4192-b13f-da0aeaac0430 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c032e85afc1e448989f8fc7086480bf4', 'request_id': 'req-1f39add0-057d-4192-b13f-da0aeaac0430'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:06.435402 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1f39add0-057d-4192-b13f-da0aeaac0430 admin admin] The token cache is being invalidate because project c032e85afc1e448989f8fc7086480bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:44:06.435805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1f39add0-057d-4192-b13f-da0aeaac0430 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:06.443042 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 887/1741] 10.222.0.59 () {58 vars in 1058 bytes} [Mon Aug 29 22:44:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:06.443780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0690f6-9283-44a7-84bc-7ea75d3ebfca None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:06.450368 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1f39add0-057d-4192-b13f-da0aeaac0430 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:06.454504 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 855/1742] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:44:06 2022] DELETE /identity/v3/projects/c032e85afc1e448989f8fc7086480bf4 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:06.456000 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39add0-057d-4192-b13f-da0aeaac0430 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:06.458885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-1f39add0-057d-4192-b13f-da0aeaac0430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:06.539105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39add0-057d-4192-b13f-da0aeaac0430 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:06.544509 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7222d2f-ee22-482f-8cd6-9a65ec732e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:06.545446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7222d2f-ee22-482f-8cd6-9a65ec732e62 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:06.545961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7222d2f-ee22-482f-8cd6-9a65ec732e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:06.546434 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7222d2f-ee22-482f-8cd6-9a65ec732e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:06.547300 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7222d2f-ee22-482f-8cd6-9a65ec732e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:06.549261 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7222d2f-ee22-482f-8cd6-9a65ec732e62 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:06.561445 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 856/1743] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:06.563356 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7222d2f-ee22-482f-8cd6-9a65ec732e62 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:06.565954 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-e7222d2f-ee22-482f-8cd6-9a65ec732e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:06.581866 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7222d2f-ee22-482f-8cd6-9a65ec732e62 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:06.587500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28687f58-8c47-4dd5-8d35-0d9817c00206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:06.588499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28687f58-8c47-4dd5-8d35-0d9817c00206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:06.589010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28687f58-8c47-4dd5-8d35-0d9817c00206 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:06.589498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28687f58-8c47-4dd5-8d35-0d9817c00206 admin admin] PATH_INFO: `/v3/users/3c3c876526f2493c857bdeaafe581eff` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:06.590873 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28687f58-8c47-4dd5-8d35-0d9817c00206 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c3c876526f2493c857bdeaafe581eff)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:06.593120 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28687f58-8c47-4dd5-8d35-0d9817c00206 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:06.609382 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0690f6-9283-44a7-84bc-7ea75d3ebfca None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:06.624440 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:06.626862 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60ccaa-3573-4ed9-bc3c-1196ba1f6197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:06.627697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60ccaa-3573-4ed9-bc3c-1196ba1f6197 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:06.627981 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60ccaa-3573-4ed9-bc3c-1196ba1f6197 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:06.628352 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60ccaa-3573-4ed9-bc3c-1196ba1f6197 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:06.628972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60ccaa-3573-4ed9-bc3c-1196ba1f6197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:06.630659 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60ccaa-3573-4ed9-bc3c-1196ba1f6197 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:06.642744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 888/1744] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:06.646027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60ccaa-3573-4ed9-bc3c-1196ba1f6197 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:06.648699 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-be60ccaa-3573-4ed9-bc3c-1196ba1f6197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:06.666585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60ccaa-3573-4ed9-bc3c-1196ba1f6197 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:06.672558 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cdf2f-5556-4bbf-901b-af843af7dfc3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:06.673304 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cdf2f-5556-4bbf-901b-af843af7dfc3 service placement] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:06.673562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cdf2f-5556-4bbf-901b-af843af7dfc3 service placement] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:06.673828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cdf2f-5556-4bbf-901b-af843af7dfc3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:06.675667 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e52cdf2f-5556-4bbf-901b-af843af7dfc3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:06.696039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52cdf2f-5556-4bbf-901b-af843af7dfc3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:06.696039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52cdf2f-5556-4bbf-901b-af843af7dfc3 service placement] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:06.707899 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 889/1745] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:06.709159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cdf2f-5556-4bbf-901b-af843af7dfc3 service placement] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:06.744812 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cdf2f-5556-4bbf-901b-af843af7dfc3 service placement] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:06.759173 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd2eec-4302-4d30-b370-ae4266b9194d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:06.759780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd2eec-4302-4d30-b370-ae4266b9194d admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:06.760059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd2eec-4302-4d30-b370-ae4266b9194d admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:06.760355 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd2eec-4302-4d30-b370-ae4266b9194d admin admin] PATH_INFO: `/v3/projects/6d721dc223fe415d8aba34f44e1b9689/users/2ee5c508f1d144f186e0cfa65e736712/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:06.798152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd2eec-4302-4d30-b370-ae4266b9194d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d721dc223fe415d8aba34f44e1b9689, user_id=2ee5c508f1d144f186e0cfa65e736712, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:06.801010 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd2eec-4302-4d30-b370-ae4266b9194d admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:06.891328 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-28687f58-8c47-4dd5-8d35-0d9817c00206 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c3c876526f2493c857bdeaafe581eff'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:06.901482 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-28687f58-8c47-4dd5-8d35-0d9817c00206 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c3c876526f2493c857bdeaafe581eff'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:06.902541 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-28687f58-8c47-4dd5-8d35-0d9817c00206 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c3c876526f2493c857bdeaafe581eff'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:06.947573 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-28687f58-8c47-4dd5-8d35-0d9817c00206 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c3c876526f2493c857bdeaafe581eff'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:06.948597 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-28687f58-8c47-4dd5-8d35-0d9817c00206 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c3c876526f2493c857bdeaafe581eff'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:07.010089 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 890/1746] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:06 2022] PUT /identity/v3/projects/6d721dc223fe415d8aba34f44e1b9689/users/2ee5c508f1d144f186e0cfa65e736712/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:07.011536 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd2eec-4302-4d30-b370-ae4266b9194d admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:07.015478 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-05cd2eec-4302-4d30-b370-ae4266b9194d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:07.038110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 857/1747] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:44:06 2022] DELETE /identity/v3/users/3c3c876526f2493c857bdeaafe581eff => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:07.044481 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28687f58-8c47-4dd5-8d35-0d9817c00206 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:07.044481 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-28687f58-8c47-4dd5-8d35-0d9817c00206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:07.160842 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd2eec-4302-4d30-b370-ae4266b9194d admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:07.182415 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28687f58-8c47-4dd5-8d35-0d9817c00206 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:07.185957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a89eec-1c1c-4413-998b-986249fdcb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:07.186771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a89eec-1c1c-4413-998b-986249fdcb04 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:07.187670 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a89eec-1c1c-4413-998b-986249fdcb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:07.188375 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a89eec-1c1c-4413-998b-986249fdcb04 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:07.189619 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a89eec-1c1c-4413-998b-986249fdcb04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:07.191943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a89eec-1c1c-4413-998b-986249fdcb04 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:07.207020 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 858/1748] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:07.212846 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a89eec-1c1c-4413-998b-986249fdcb04 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:07.215152 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-00a89eec-1c1c-4413-998b-986249fdcb04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:07.242963 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a89eec-1c1c-4413-998b-986249fdcb04 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:07.252337 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69b7da-d6be-4567-a069-debb6ea0c368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:07.253154 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69b7da-d6be-4567-a069-debb6ea0c368 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:07.253633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69b7da-d6be-4567-a069-debb6ea0c368 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:07.257152 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69b7da-d6be-4567-a069-debb6ea0c368 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:07.257152 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b69b7da-d6be-4567-a069-debb6ea0c368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:07.257349 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b69b7da-d6be-4567-a069-debb6ea0c368 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:07.279523 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 859/1749] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:07.280636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69b7da-d6be-4567-a069-debb6ea0c368 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:07.293073 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69b7da-d6be-4567-a069-debb6ea0c368 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:07.301463 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727c93d-e9f4-4cfd-8543-363f4882130b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:07.302305 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727c93d-e9f4-4cfd-8543-363f4882130b admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:07.302676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727c93d-e9f4-4cfd-8543-363f4882130b admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:07.303041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727c93d-e9f4-4cfd-8543-363f4882130b admin admin] PATH_INFO: `/v3/projects/623fb48adc4147f8ab2c7037278eb2b5/users/2592a7be71ec4f35bb2a2b5fba8dab56/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:07.328807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f727c93d-e9f4-4cfd-8543-363f4882130b admin admin] RBAC: Authorizing `identity:create_grant(project_id=623fb48adc4147f8ab2c7037278eb2b5, user_id=2592a7be71ec4f35bb2a2b5fba8dab56, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:07.331496 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f727c93d-e9f4-4cfd-8543-363f4882130b admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:07.341759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a32d3c-f39d-4204-a328-458838cb06fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:07.342382 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a32d3c-f39d-4204-a328-458838cb06fb service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:07.342661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a32d3c-f39d-4204-a328-458838cb06fb service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:07.342932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a32d3c-f39d-4204-a328-458838cb06fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:07.344769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-89a32d3c-f39d-4204-a328-458838cb06fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:07.434098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a32d3c-f39d-4204-a328-458838cb06fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:07.436781 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a32d3c-f39d-4204-a328-458838cb06fb service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:07.464507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 860/1751] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:07 2022] PUT /identity/v3/projects/623fb48adc4147f8ab2c7037278eb2b5/users/2592a7be71ec4f35bb2a2b5fba8dab56/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:07.464507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727c93d-e9f4-4cfd-8543-363f4882130b admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:07.465958 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 891/1751] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:07.468367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4a455-9655-4c32-b7ba-2b2aaa95da61 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:07.468929 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4a455-9655-4c32-b7ba-2b2aaa95da61 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:07.469337 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4a455-9655-4c32-b7ba-2b2aaa95da61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:07.504676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-72e4a455-9655-4c32-b7ba-2b2aaa95da61 None None] MFA Rules not processed for user `2592a7be71ec4f35bb2a2b5fba8dab56`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:07.506374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-72e4a455-9655-4c32-b7ba-2b2aaa95da61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:07.582741 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727c93d-e9f4-4cfd-8543-363f4882130b admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:07.589729 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26608a7f-ab9f-47fb-9f22-0e905a4b0d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:07.591012 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26608a7f-ab9f-47fb-9f22-0e905a4b0d30 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:07.592075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26608a7f-ab9f-47fb-9f22-0e905a4b0d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:07.592625 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26608a7f-ab9f-47fb-9f22-0e905a4b0d30 admin admin] PATH_INFO: `/v3/projects/6d721dc223fe415d8aba34f44e1b9689/users/2ee5c508f1d144f186e0cfa65e736712/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:07.596222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26608a7f-ab9f-47fb-9f22-0e905a4b0d30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d721dc223fe415d8aba34f44e1b9689, user_id=2ee5c508f1d144f186e0cfa65e736712, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:07.600092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26608a7f-ab9f-47fb-9f22-0e905a4b0d30 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:07.627043 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 861/1752] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:07 2022] PUT /identity/v3/projects/6d721dc223fe415d8aba34f44e1b9689/users/2ee5c508f1d144f186e0cfa65e736712/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:07.629198 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9df206-75a5-48b5-884d-a037c08651c9 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:07.629717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9df206-75a5-48b5-884d-a037c08651c9 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:07.630187 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9df206-75a5-48b5-884d-a037c08651c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:07.677391 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-5d9df206-75a5-48b5-884d-a037c08651c9 None None] MFA Rules not processed for user `2ee5c508f1d144f186e0cfa65e736712`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:07.680352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-5d9df206-75a5-48b5-884d-a037c08651c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:07.747097 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 892/1753] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:07.757034 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e4a455-9655-4c32-b7ba-2b2aaa95da61 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:07.800751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e4a455-9655-4c32-b7ba-2b2aaa95da61 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:07.912073 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 862/1754] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:07 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:07.947467 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9df206-75a5-48b5-884d-a037c08651c9 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:07.949018 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-5d9df206-75a5-48b5-884d-a037c08651c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:08.058338 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9df206-75a5-48b5-884d-a037c08651c9 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:08.066870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df9e24-f508-4838-b7da-3d2512cbf0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:08.078923 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df9e24-f508-4838-b7da-3d2512cbf0eb admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:08.078923 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df9e24-f508-4838-b7da-3d2512cbf0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:08.078923 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df9e24-f508-4838-b7da-3d2512cbf0eb admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:08.078923 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20df9e24-f508-4838-b7da-3d2512cbf0eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:08.079304 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d2fb5-11e3-42fb-8ff7-cb52656904d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:08.079304 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d2fb5-11e3-42fb-8ff7-cb52656904d5 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:08.079304 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d2fb5-11e3-42fb-8ff7-cb52656904d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:08.079304 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d2fb5-11e3-42fb-8ff7-cb52656904d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:08.080310 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20df9e24-f508-4838-b7da-3d2512cbf0eb admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:08.119198 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d2fb5-11e3-42fb-8ff7-cb52656904d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:08.122651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d2fb5-11e3-42fb-8ff7-cb52656904d5 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:08.171147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 863/1755] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:07 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:08.171147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d2fb5-11e3-42fb-8ff7-cb52656904d5 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:08.171147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a73d2fb5-11e3-42fb-8ff7-cb52656904d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:08.227203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d2fb5-11e3-42fb-8ff7-cb52656904d5 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:08.254641 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 893/1756] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:44:07 2022] POST /identity/v3/projects => generated 383 bytes in 502 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:08.260121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df9e24-f508-4838-b7da-3d2512cbf0eb admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:08.265497 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e21d1b-df46-4ad2-88f4-7c0df50e97aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:08.268405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e21d1b-df46-4ad2-88f4-7c0df50e97aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:08.268683 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e21d1b-df46-4ad2-88f4-7c0df50e97aa admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:08.268953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e21d1b-df46-4ad2-88f4-7c0df50e97aa admin admin] PATH_INFO: `/v3/projects/1ec1b40c800247c3879b9feadf7f3925` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:08.288502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e21d1b-df46-4ad2-88f4-7c0df50e97aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ec1b40c800247c3879b9feadf7f3925)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:08.288502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e21d1b-df46-4ad2-88f4-7c0df50e97aa admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:08.436841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df9e24-f508-4838-b7da-3d2512cbf0eb admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:08.442154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40c016-fe53-4bff-939d-c6f1040a381b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:08.442769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40c016-fe53-4bff-939d-c6f1040a381b admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:08.443099 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40c016-fe53-4bff-939d-c6f1040a381b admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:08.443375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40c016-fe53-4bff-939d-c6f1040a381b admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:08.444066 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40c016-fe53-4bff-939d-c6f1040a381b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:08.446196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40c016-fe53-4bff-939d-c6f1040a381b admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:08.479891 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-07e21d1b-df46-4ad2-88f4-7c0df50e97aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ec1b40c800247c3879b9feadf7f3925', 'request_id': 'req-07e21d1b-df46-4ad2-88f4-7c0df50e97aa'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:08.554911 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 894/1757] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:08 2022] POST /identity/v3/users => generated 298 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:08.556439 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40c016-fe53-4bff-939d-c6f1040a381b admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:08.579265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40c016-fe53-4bff-939d-c6f1040a381b admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:08.594484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-07e21d1b-df46-4ad2-88f4-7c0df50e97aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ec1b40c800247c3879b9feadf7f3925', 'request_id': 'req-07e21d1b-df46-4ad2-88f4-7c0df50e97aa'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:08.603998 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-07e21d1b-df46-4ad2-88f4-7c0df50e97aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ec1b40c800247c3879b9feadf7f3925', 'request_id': 'req-07e21d1b-df46-4ad2-88f4-7c0df50e97aa'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:08.622166 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-07e21d1b-df46-4ad2-88f4-7c0df50e97aa admin admin] The token cache is being invalidate because project 1ec1b40c800247c3879b9feadf7f3925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:44:08.622782 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-07e21d1b-df46-4ad2-88f4-7c0df50e97aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:08.623657 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-07e21d1b-df46-4ad2-88f4-7c0df50e97aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:08.626877 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 864/1758] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:44:08 2022] DELETE /identity/v3/projects/1ec1b40c800247c3879b9feadf7f3925 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:08.633522 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c266c-375a-40ce-87aa-1c9a1fb5a9ad None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:08.634366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c266c-375a-40ce-87aa-1c9a1fb5a9ad None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:08.634852 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c266c-375a-40ce-87aa-1c9a1fb5a9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:08.676363 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6df69f-4d70-4123-a153-f7db2690ce2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:08.677116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6df69f-4d70-4123-a153-f7db2690ce2a admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:08.677638 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6df69f-4d70-4123-a153-f7db2690ce2a admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:08.677987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6df69f-4d70-4123-a153-f7db2690ce2a admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:08.679350 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6df69f-4d70-4123-a153-f7db2690ce2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:08.681207 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6df69f-4d70-4123-a153-f7db2690ce2a admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:08.691321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-324c266c-375a-40ce-87aa-1c9a1fb5a9ad None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:08.695808 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-324c266c-375a-40ce-87aa-1c9a1fb5a9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:08.706110 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 895/1759] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:08.707184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6df69f-4d70-4123-a153-f7db2690ce2a admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:08.713774 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 865/1760] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:08.715012 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c266c-375a-40ce-87aa-1c9a1fb5a9ad None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:08.716792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-4c6df69f-4d70-4123-a153-f7db2690ce2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:08.718013 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-324c266c-375a-40ce-87aa-1c9a1fb5a9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:08.732276 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c266c-375a-40ce-87aa-1c9a1fb5a9ad None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:08.739296 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c2ae8e-ed34-44a7-ae6c-eeb3d9db207e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:08.740391 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2ae8e-ed34-44a7-ae6c-eeb3d9db207e admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:08.740875 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2ae8e-ed34-44a7-ae6c-eeb3d9db207e admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:08.741977 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2ae8e-ed34-44a7-ae6c-eeb3d9db207e admin admin] PATH_INFO: `/v3/projects/b348bd4ebe5b49cbbb393e862a8389ab/users/dfcc874269b8406799d07c4b2fd717fa/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:08.746419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:08.772005 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c2ae8e-ed34-44a7-ae6c-eeb3d9db207e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b348bd4ebe5b49cbbb393e862a8389ab, user_id=dfcc874269b8406799d07c4b2fd717fa, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:08.776283 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c2ae8e-ed34-44a7-ae6c-eeb3d9db207e admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:08.804987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6df69f-4d70-4123-a153-f7db2690ce2a admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:08.812041 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a25c0c-ce54-4340-9b3d-3a6693cdb9d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:08.812888 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a25c0c-ce54-4340-9b3d-3a6693cdb9d6 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:08.813163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a25c0c-ce54-4340-9b3d-3a6693cdb9d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:08.813513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a25c0c-ce54-4340-9b3d-3a6693cdb9d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:08.815484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e3a25c0c-ce54-4340-9b3d-3a6693cdb9d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:08.907138 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a25c0c-ce54-4340-9b3d-3a6693cdb9d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:08.908608 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a25c0c-ce54-4340-9b3d-3a6693cdb9d6 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:08.922734 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 896/1761] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:08 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:08.924577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75179f96-7edb-4a2d-aaf7-0fa522fbcbf3 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:08.924969 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75179f96-7edb-4a2d-aaf7-0fa522fbcbf3 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:08.925343 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75179f96-7edb-4a2d-aaf7-0fa522fbcbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:08.941112 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 866/1762] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:08 2022] PUT /identity/v3/projects/b348bd4ebe5b49cbbb393e862a8389ab/users/dfcc874269b8406799d07c4b2fd717fa/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:08.942396 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c2ae8e-ed34-44a7-ae6c-eeb3d9db207e admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:08.950665 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-75179f96-7edb-4a2d-aaf7-0fa522fbcbf3 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:08.952051 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-75179f96-7edb-4a2d-aaf7-0fa522fbcbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:09.090693 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c2ae8e-ed34-44a7-ae6c-eeb3d9db207e admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:09.098924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 897/1763] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:09.100216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75179f96-7edb-4a2d-aaf7-0fa522fbcbf3 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:09.107241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7daac83-d36a-4d96-bf68-ac4fc1c66a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:09.108380 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7daac83-d36a-4d96-bf68-ac4fc1c66a2b admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:09.109456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7daac83-d36a-4d96-bf68-ac4fc1c66a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:09.109736 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7daac83-d36a-4d96-bf68-ac4fc1c66a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:09.113189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7daac83-d36a-4d96-bf68-ac4fc1c66a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:09.115251 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7daac83-d36a-4d96-bf68-ac4fc1c66a2b admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:09.126112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75179f96-7edb-4a2d-aaf7-0fa522fbcbf3 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:09.128113 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 867/1764] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:09.132900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7daac83-d36a-4d96-bf68-ac4fc1c66a2b admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:09.145090 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1265a-4cc1-4e92-9118-5aa747692986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:09.145300 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1265a-4cc1-4e92-9118-5aa747692986 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:09.149425 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1265a-4cc1-4e92-9118-5aa747692986 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:09.149822 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1265a-4cc1-4e92-9118-5aa747692986 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:09.152925 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7daac83-d36a-4d96-bf68-ac4fc1c66a2b admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:09.153051 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1265a-4cc1-4e92-9118-5aa747692986 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:44:09.153051 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1265a-4cc1-4e92-9118-5aa747692986 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:09.153051 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1265a-4cc1-4e92-9118-5aa747692986 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:09.157515 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c3dc4-cfe4-4c24-b9e2-ba5b6f7e92c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:09.158687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c3dc4-cfe4-4c24-b9e2-ba5b6f7e92c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:09.159014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c3dc4-cfe4-4c24-b9e2-ba5b6f7e92c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:09.159290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c3dc4-cfe4-4c24-b9e2-ba5b6f7e92c4 admin admin] PATH_INFO: `/v3/projects/b348bd4ebe5b49cbbb393e862a8389ab/users/dfcc874269b8406799d07c4b2fd717fa/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:09.161940 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c3dc4-cfe4-4c24-b9e2-ba5b6f7e92c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b348bd4ebe5b49cbbb393e862a8389ab, user_id=dfcc874269b8406799d07c4b2fd717fa, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:09.166088 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c3dc4-cfe4-4c24-b9e2-ba5b6f7e92c4 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:09.181266 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 898/1765] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:44:09.199417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1265a-4cc1-4e92-9118-5aa747692986 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:09.217869 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 868/1766] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:09 2022] PUT /identity/v3/projects/b348bd4ebe5b49cbbb393e862a8389ab/users/dfcc874269b8406799d07c4b2fd717fa/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:09.233037 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1265a-4cc1-4e92-9118-5aa747692986 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:09.243088 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e602b-be75-40ac-bca6-14b182104236 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:09.243373 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e602b-be75-40ac-bca6-14b182104236 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:09.243646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e602b-be75-40ac-bca6-14b182104236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:09.298101 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-975e602b-be75-40ac-bca6-14b182104236 None None] MFA Rules not processed for user `dfcc874269b8406799d07c4b2fd717fa`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:09.301934 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-975e602b-be75-40ac-bca6-14b182104236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:09.305914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f7f3d-43ff-4569-95f2-d83ff742a4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:09.306762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f7f3d-43ff-4569-95f2-d83ff742a4e3 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:09.307251 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f7f3d-43ff-4569-95f2-d83ff742a4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:09.307726 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f7f3d-43ff-4569-95f2-d83ff742a4e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:09.308759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f7f3d-43ff-4569-95f2-d83ff742a4e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:09.311126 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f7f3d-43ff-4569-95f2-d83ff742a4e3 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:09.381737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 869/1767] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:09.393884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e602b-be75-40ac-bca6-14b182104236 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:09.395497 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-975e602b-be75-40ac-bca6-14b182104236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:09.482762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e602b-be75-40ac-bca6-14b182104236 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:09.488654 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3762eb-73c7-4847-b86e-d474821ae1f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:09.489275 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3762eb-73c7-4847-b86e-d474821ae1f7 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:09.489880 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3762eb-73c7-4847-b86e-d474821ae1f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:09.490415 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3762eb-73c7-4847-b86e-d474821ae1f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:09.503030 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3762eb-73c7-4847-b86e-d474821ae1f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:09.504935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3762eb-73c7-4847-b86e-d474821ae1f7 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:09.521712 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 870/1768] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:44:09.632718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 899/1769] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:44:09 2022] POST /identity/v3/projects => generated 367 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:09.635927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3762eb-73c7-4847-b86e-d474821ae1f7 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:09.648323 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3762eb-73c7-4847-b86e-d474821ae1f7 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:09.715882 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f7f3d-43ff-4569-95f2-d83ff742a4e3 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:09.718184 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d529c-0835-4d94-be4d-60edef3f00d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:09.718896 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d529c-0835-4d94-be4d-60edef3f00d2 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:09.719286 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d529c-0835-4d94-be4d-60edef3f00d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:09.719629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d529c-0835-4d94-be4d-60edef3f00d2 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:09.720402 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d529c-0835-4d94-be4d-60edef3f00d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:09.723436 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d529c-0835-4d94-be4d-60edef3f00d2 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:09.846164 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f7f3d-43ff-4569-95f2-d83ff742a4e3 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:09.858429 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c6853-bd05-44a2-a330-4f6622a1ce7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:09.859110 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c6853-bd05-44a2-a330-4f6622a1ce7a service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:09.859386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c6853-bd05-44a2-a330-4f6622a1ce7a service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:09.859658 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c6853-bd05-44a2-a330-4f6622a1ce7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:09.867836 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-cd5c6853-bd05-44a2-a330-4f6622a1ce7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:09.936866 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 871/1770] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:09 2022] POST /identity/v3/users => generated 284 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:09.938310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d529c-0835-4d94-be4d-60edef3f00d2 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:09.938514 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c6853-bd05-44a2-a330-4f6622a1ce7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:09.940339 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c6853-bd05-44a2-a330-4f6622a1ce7a service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:09.951791 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d529c-0835-4d94-be4d-60edef3f00d2 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:09.960623 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685e722-cf1a-4d17-b05b-162fe58a9d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:09.961338 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685e722-cf1a-4d17-b05b-162fe58a9d20 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:09.962900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685e722-cf1a-4d17-b05b-162fe58a9d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:09.962900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685e722-cf1a-4d17-b05b-162fe58a9d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:09.962900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4685e722-cf1a-4d17-b05b-162fe58a9d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:09.965117 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4685e722-cf1a-4d17-b05b-162fe58a9d20 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:09.966281 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 900/1771] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:44:09.981083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 872/1772] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:09.985116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c6853-bd05-44a2-a330-4f6622a1ce7a service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:09.997750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c6853-bd05-44a2-a330-4f6622a1ce7a service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:10.003301 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8355d2-db34-422e-bd50-932ddcc7459e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:10.003800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8355d2-db34-422e-bd50-932ddcc7459e admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:10.004076 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8355d2-db34-422e-bd50-932ddcc7459e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:10.004377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8355d2-db34-422e-bd50-932ddcc7459e admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:10.005020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8355d2-db34-422e-bd50-932ddcc7459e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:10.007005 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8355d2-db34-422e-bd50-932ddcc7459e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:10.023648 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 901/1773] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:44:10.029912 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685e722-cf1a-4d17-b05b-162fe58a9d20 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:10.054175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685e722-cf1a-4d17-b05b-162fe58a9d20 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:10.064966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6016cfb-de02-492a-9cd6-2643efeda460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:10.065905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6016cfb-de02-492a-9cd6-2643efeda460 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:10.066192 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6016cfb-de02-492a-9cd6-2643efeda460 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:10.066468 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6016cfb-de02-492a-9cd6-2643efeda460 admin admin] PATH_INFO: `/v3/projects/aea5e328b0d342188f0d75a05f2edc3f/users/4267fbbf0c01465c99320c848e71fda0/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:10.090407 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6016cfb-de02-492a-9cd6-2643efeda460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aea5e328b0d342188f0d75a05f2edc3f, user_id=4267fbbf0c01465c99320c848e71fda0, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:10.093424 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6016cfb-de02-492a-9cd6-2643efeda460 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:10.235658 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 873/1774] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:10 2022] PUT /identity/v3/projects/aea5e328b0d342188f0d75a05f2edc3f/users/4267fbbf0c01465c99320c848e71fda0/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:10.249937 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dfb6d-5da7-4736-85b5-ceb3296ad1f9 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:10.249937 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dfb6d-5da7-4736-85b5-ceb3296ad1f9 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:10.249937 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dfb6d-5da7-4736-85b5-ceb3296ad1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:10.307962 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-3a9dfb6d-5da7-4736-85b5-ceb3296ad1f9 None None] MFA Rules not processed for user `4267fbbf0c01465c99320c848e71fda0`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:10.312030 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3a9dfb6d-5da7-4736-85b5-ceb3296ad1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:10.357251 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:10.473934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 902/1775] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:10.478797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6016cfb-de02-492a-9cd6-2643efeda460 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:10.567430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6016cfb-de02-492a-9cd6-2643efeda460 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:10.573667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9175ab9-1421-491e-b61f-775bd012f28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:10.574663 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9175ab9-1421-491e-b61f-775bd012f28d admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:10.576540 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9175ab9-1421-491e-b61f-775bd012f28d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:10.577003 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9175ab9-1421-491e-b61f-775bd012f28d admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:10.577878 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9175ab9-1421-491e-b61f-775bd012f28d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:10.584171 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9175ab9-1421-491e-b61f-775bd012f28d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:10.692320 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 874/1776] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:44:10 2022] POST /identity/v3/projects => generated 367 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:10.696755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9dfb6d-5da7-4736-85b5-ceb3296ad1f9 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:10.713675 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9dfb6d-5da7-4736-85b5-ceb3296ad1f9 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:10.946476 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94e6d3-642a-4959-a103-aa21e5480ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:10.964929 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94e6d3-642a-4959-a103-aa21e5480ed9 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:10.965541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94e6d3-642a-4959-a103-aa21e5480ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:10.966500 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94e6d3-642a-4959-a103-aa21e5480ed9 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:10.967858 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a94e6d3-642a-4959-a103-aa21e5480ed9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:10.971083 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a94e6d3-642a-4959-a103-aa21e5480ed9 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:11.093160 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 903/1777] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:10 2022] POST /identity/v3/users => generated 290 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:11.097243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9175ab9-1421-491e-b61f-775bd012f28d admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:11.111642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9175ab9-1421-491e-b61f-775bd012f28d admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:11.117561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750a61b-63de-4c82-b5c8-3a2a441520d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:11.118462 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750a61b-63de-4c82-b5c8-3a2a441520d6 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:11.118836 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750a61b-63de-4c82-b5c8-3a2a441520d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:11.119126 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750a61b-63de-4c82-b5c8-3a2a441520d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:11.119771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750a61b-63de-4c82-b5c8-3a2a441520d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:11.121483 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750a61b-63de-4c82-b5c8-3a2a441520d6 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:11.134027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 875/1778] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:11.138291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94e6d3-642a-4959-a103-aa21e5480ed9 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:11.148582 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94e6d3-642a-4959-a103-aa21e5480ed9 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:11.153554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643668e-aa47-4455-8e70-8d0cec7d7b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:11.154227 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643668e-aa47-4455-8e70-8d0cec7d7b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:11.154504 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643668e-aa47-4455-8e70-8d0cec7d7b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:11.154775 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643668e-aa47-4455-8e70-8d0cec7d7b26 admin admin] PATH_INFO: `/v3/projects/93306ba6fcce457d9a839355ad89a897/users/ae9aee54902e48feaff7903d03aebb3e/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:11.171469 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643668e-aa47-4455-8e70-8d0cec7d7b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93306ba6fcce457d9a839355ad89a897, user_id=ae9aee54902e48feaff7903d03aebb3e, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:11.174815 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643668e-aa47-4455-8e70-8d0cec7d7b26 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:11.376825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 904/1779] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:11 2022] PUT /identity/v3/projects/93306ba6fcce457d9a839355ad89a897/users/ae9aee54902e48feaff7903d03aebb3e/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:44:11.377683 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750a61b-63de-4c82-b5c8-3a2a441520d6 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:11.387850 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750a61b-63de-4c82-b5c8-3a2a441520d6 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:11.513733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5f3e3-7a45-4780-8730-e45cafee4d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:11.514664 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5f3e3-7a45-4780-8730-e45cafee4d6a admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:11.515109 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5f3e3-7a45-4780-8730-e45cafee4d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:11.515577 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5f3e3-7a45-4780-8730-e45cafee4d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:11.517820 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce5f3e3-7a45-4780-8730-e45cafee4d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:11.520788 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce5f3e3-7a45-4780-8730-e45cafee4d6a admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:11.535498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 876/1780] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:44:11.540636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643668e-aa47-4455-8e70-8d0cec7d7b26 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:11.553539 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643668e-aa47-4455-8e70-8d0cec7d7b26 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:11.559620 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc64c8-7028-428f-ac85-5c08180f5891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:11.560328 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc64c8-7028-428f-ac85-5c08180f5891 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:11.560615 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc64c8-7028-428f-ac85-5c08180f5891 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:11.560891 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc64c8-7028-428f-ac85-5c08180f5891 admin admin] PATH_INFO: `/v3/projects/93306ba6fcce457d9a839355ad89a897/users/ae9aee54902e48feaff7903d03aebb3e/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:11.562873 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc64c8-7028-428f-ac85-5c08180f5891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93306ba6fcce457d9a839355ad89a897, user_id=ae9aee54902e48feaff7903d03aebb3e, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:11.566890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc64c8-7028-428f-ac85-5c08180f5891 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:11.578614 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 905/1781] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:11 2022] PUT /identity/v3/projects/93306ba6fcce457d9a839355ad89a897/users/ae9aee54902e48feaff7903d03aebb3e/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:44:11.592182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5dfce-9aaf-4c9a-8b95-643eebe8983b None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:11.592501 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5dfce-9aaf-4c9a-8b95-643eebe8983b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:11.592798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5dfce-9aaf-4c9a-8b95-643eebe8983b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:11.635986 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-8bb5dfce-9aaf-4c9a-8b95-643eebe8983b None None] MFA Rules not processed for user `ae9aee54902e48feaff7903d03aebb3e`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:11.637360 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-8bb5dfce-9aaf-4c9a-8b95-643eebe8983b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:11.752710 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 877/1782] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:11.774630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc64c8-7028-428f-ac85-5c08180f5891 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:11.947208 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:11.967670 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc64c8-7028-428f-ac85-5c08180f5891 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:11.986627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7df433-ce72-4fa7-adf9-669e1d22e67a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:11.987266 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7df433-ce72-4fa7-adf9-669e1d22e67a service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:12.001893 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7df433-ce72-4fa7-adf9-669e1d22e67a service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:12.001893 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7df433-ce72-4fa7-adf9-669e1d22e67a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:12.031432 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7df433-ce72-4fa7-adf9-669e1d22e67a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:12.033452 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7df433-ce72-4fa7-adf9-669e1d22e67a service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:12.070207 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 906/1783] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:12.191598 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5dfce-9aaf-4c9a-8b95-643eebe8983b None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:12.197180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-8bb5dfce-9aaf-4c9a-8b95-643eebe8983b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:12.343120 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5dfce-9aaf-4c9a-8b95-643eebe8983b None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:12.358773 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097f3e0-fb55-46a2-ac09-34054919accf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:12.365467 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097f3e0-fb55-46a2-ac09-34054919accf admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:12.365820 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097f3e0-fb55-46a2-ac09-34054919accf admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:12.366299 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097f3e0-fb55-46a2-ac09-34054919accf admin admin] PATH_INFO: `/v3/users/5a454af87e5f427487019b237aab286b` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:12.367868 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097f3e0-fb55-46a2-ac09-34054919accf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a454af87e5f427487019b237aab286b)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:12.370184 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097f3e0-fb55-46a2-ac09-34054919accf admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:12.389234 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7df433-ce72-4fa7-adf9-669e1d22e67a service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:12.401106 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7df433-ce72-4fa7-adf9-669e1d22e67a service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:12.419550 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01a93e-8ed5-4ca5-bc0e-427d7ef212d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:12.420989 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01a93e-8ed5-4ca5-bc0e-427d7ef212d0 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:12.421777 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01a93e-8ed5-4ca5-bc0e-427d7ef212d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:12.422669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01a93e-8ed5-4ca5-bc0e-427d7ef212d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:12.579791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01a93e-8ed5-4ca5-bc0e-427d7ef212d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:12.598872 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01a93e-8ed5-4ca5-bc0e-427d7ef212d0 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:12.611757 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 907/1784] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:44:12.630984 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-9097f3e0-fb55-46a2-ac09-34054919accf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a454af87e5f427487019b237aab286b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:12.641288 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-9097f3e0-fb55-46a2-ac09-34054919accf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a454af87e5f427487019b237aab286b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:12.642666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-9097f3e0-fb55-46a2-ac09-34054919accf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a454af87e5f427487019b237aab286b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:12.688298 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-9097f3e0-fb55-46a2-ac09-34054919accf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a454af87e5f427487019b237aab286b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:12.690496 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-9097f3e0-fb55-46a2-ac09-34054919accf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a454af87e5f427487019b237aab286b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:12.797881 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 878/1785] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:44:12 2022] DELETE /identity/v3/users/5a454af87e5f427487019b237aab286b => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:12.804367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6f7a2-aeeb-4f97-967c-3ce813574b3c None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:12.805209 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6f7a2-aeeb-4f97-967c-3ce813574b3c None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:12.806125 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6f7a2-aeeb-4f97-967c-3ce813574b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:12.857775 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-04f6f7a2-aeeb-4f97-967c-3ce813574b3c None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:12.860940 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-04f6f7a2-aeeb-4f97-967c-3ce813574b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:12.987899 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 908/1786] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:13.001837 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097f3e0-fb55-46a2-ac09-34054919accf admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:13.003881 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-9097f3e0-fb55-46a2-ac09-34054919accf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:13.014927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:13.086535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097f3e0-fb55-46a2-ac09-34054919accf admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:13.091082 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c144aa5-cb0e-41ad-9b66-62fd88a338db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:13.091807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c144aa5-cb0e-41ad-9b66-62fd88a338db service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:13.092213 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c144aa5-cb0e-41ad-9b66-62fd88a338db service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:13.092581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c144aa5-cb0e-41ad-9b66-62fd88a338db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:13.103294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c144aa5-cb0e-41ad-9b66-62fd88a338db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:13.104930 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c144aa5-cb0e-41ad-9b66-62fd88a338db service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:13.117115 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 879/1787] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:13.659577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6f7a2-aeeb-4f97-967c-3ce813574b3c None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:13.661879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-04f6f7a2-aeeb-4f97-967c-3ce813574b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:13.677005 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6f7a2-aeeb-4f97-967c-3ce813574b3c None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:13.682066 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356637e9-d9af-4282-8bd1-0d77f604ab6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:13.683478 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356637e9-d9af-4282-8bd1-0d77f604ab6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:13.683952 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356637e9-d9af-4282-8bd1-0d77f604ab6c admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:13.684356 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356637e9-d9af-4282-8bd1-0d77f604ab6c admin admin] PATH_INFO: `/v3/projects/ffd51e41f76e4ff0a89de9365049e149` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:13.685810 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356637e9-d9af-4282-8bd1-0d77f604ab6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffd51e41f76e4ff0a89de9365049e149)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:13.688078 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356637e9-d9af-4282-8bd1-0d77f604ab6c admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:13.825271 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-356637e9-d9af-4282-8bd1-0d77f604ab6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffd51e41f76e4ff0a89de9365049e149', 'request_id': 'req-356637e9-d9af-4282-8bd1-0d77f604ab6c'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:13.836029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-356637e9-d9af-4282-8bd1-0d77f604ab6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffd51e41f76e4ff0a89de9365049e149', 'request_id': 'req-356637e9-d9af-4282-8bd1-0d77f604ab6c'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:13.846525 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-356637e9-d9af-4282-8bd1-0d77f604ab6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffd51e41f76e4ff0a89de9365049e149', 'request_id': 'req-356637e9-d9af-4282-8bd1-0d77f604ab6c'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:13.915017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-356637e9-d9af-4282-8bd1-0d77f604ab6c admin admin] The token cache is being invalidate because project ffd51e41f76e4ff0a89de9365049e149 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:44:13.915755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-356637e9-d9af-4282-8bd1-0d77f604ab6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:13.919286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-356637e9-d9af-4282-8bd1-0d77f604ab6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:13.921819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 909/1788] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:44:13 2022] DELETE /identity/v3/projects/ffd51e41f76e4ff0a89de9365049e149 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:13.926919 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e61e6-9bd5-470a-836a-9e6d67ebaf7f None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:13.927626 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e61e6-9bd5-470a-836a-9e6d67ebaf7f None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:13.927626 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e61e6-9bd5-470a-836a-9e6d67ebaf7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:13.969461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-414e61e6-9bd5-470a-836a-9e6d67ebaf7f None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:13.972408 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-414e61e6-9bd5-470a-836a-9e6d67ebaf7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:13.998688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:14.101210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 880/1789] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:14.107932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9c661-e23c-4feb-ae20-d4fef3574c06 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:14.108245 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9c661-e23c-4feb-ae20-d4fef3574c06 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:14.108516 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9c661-e23c-4feb-ae20-d4fef3574c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:14.134669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-c8b9c661-e23c-4feb-ae20-d4fef3574c06 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:14.137202 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c8b9c661-e23c-4feb-ae20-d4fef3574c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:14.145830 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 910/1790] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:14.150756 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e61e6-9bd5-470a-836a-9e6d67ebaf7f None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:14.162522 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e61e6-9bd5-470a-836a-9e6d67ebaf7f None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:14.175366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a2820-79fd-4f46-b4d2-01c847d6f6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:14.176021 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a2820-79fd-4f46-b4d2-01c847d6f6ad admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:14.176826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a2820-79fd-4f46-b4d2-01c847d6f6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:14.177154 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a2820-79fd-4f46-b4d2-01c847d6f6ad admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:14.179076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a2820-79fd-4f46-b4d2-01c847d6f6ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:44:14.179076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a2820-79fd-4f46-b4d2-01c847d6f6ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:14.180895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a2820-79fd-4f46-b4d2-01c847d6f6ad admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:14.199339 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 881/1791] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:44:14.200762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9c661-e23c-4feb-ae20-d4fef3574c06 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:14.211777 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9c661-e23c-4feb-ae20-d4fef3574c06 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:14.217456 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065482be-9abc-4528-bbf7-418d84e89201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:14.218112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065482be-9abc-4528-bbf7-418d84e89201 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:14.218390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065482be-9abc-4528-bbf7-418d84e89201 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:14.218683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065482be-9abc-4528-bbf7-418d84e89201 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:14.219328 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065482be-9abc-4528-bbf7-418d84e89201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:14.221315 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065482be-9abc-4528-bbf7-418d84e89201 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:14.360370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 911/1792] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:44:14 2022] POST /identity/v3/projects => generated 355 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:14.375331 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a2820-79fd-4f46-b4d2-01c847d6f6ad admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:14.387963 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a2820-79fd-4f46-b4d2-01c847d6f6ad admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:14.450212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c291f-b02b-4840-bbcd-ceadf015f7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:14.450965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c291f-b02b-4840-bbcd-ceadf015f7bd admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:14.451343 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c291f-b02b-4840-bbcd-ceadf015f7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:14.451714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c291f-b02b-4840-bbcd-ceadf015f7bd admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:14.452577 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c291f-b02b-4840-bbcd-ceadf015f7bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:14.455107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c291f-b02b-4840-bbcd-ceadf015f7bd admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:14.572468 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 882/1793] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:14 2022] POST /identity/v3/users => generated 278 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:14.576847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065482be-9abc-4528-bbf7-418d84e89201 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:14.589664 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065482be-9abc-4528-bbf7-418d84e89201 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:14.594747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd3479-cfbc-4b1b-9d35-0a6f5a5c5354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:14.595347 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd3479-cfbc-4b1b-9d35-0a6f5a5c5354 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:14.595619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd3479-cfbc-4b1b-9d35-0a6f5a5c5354 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:14.595882 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd3479-cfbc-4b1b-9d35-0a6f5a5c5354 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:14.596559 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd3479-cfbc-4b1b-9d35-0a6f5a5c5354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:14.598274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd3479-cfbc-4b1b-9d35-0a6f5a5c5354 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:14.610530 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 912/1794] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:14.614927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c291f-b02b-4840-bbcd-ceadf015f7bd admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:14.635188 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c291f-b02b-4840-bbcd-ceadf015f7bd admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:14.641320 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13f171-94a2-4554-ae84-9c4bd00317e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:14.641974 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13f171-94a2-4554-ae84-9c4bd00317e2 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:14.642249 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13f171-94a2-4554-ae84-9c4bd00317e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:14.642519 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13f171-94a2-4554-ae84-9c4bd00317e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:14.643172 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13f171-94a2-4554-ae84-9c4bd00317e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:14.644867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13f171-94a2-4554-ae84-9c4bd00317e2 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:14.663988 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 883/1795] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:14.665002 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd3479-cfbc-4b1b-9d35-0a6f5a5c5354 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:14.675051 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd3479-cfbc-4b1b-9d35-0a6f5a5c5354 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:14.680838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd6fff-423f-4d3b-8bf8-2aaa3a1879a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:14.681602 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd6fff-423f-4d3b-8bf8-2aaa3a1879a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:14.682010 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd6fff-423f-4d3b-8bf8-2aaa3a1879a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:14.682285 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd6fff-423f-4d3b-8bf8-2aaa3a1879a9 admin admin] PATH_INFO: `/v3/projects/730a8bf0156545a2b8a0d926002bf809/users/92175ee3c02d4c919bfdbb26d5e4a4de/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:14.698686 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd6fff-423f-4d3b-8bf8-2aaa3a1879a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=730a8bf0156545a2b8a0d926002bf809, user_id=92175ee3c02d4c919bfdbb26d5e4a4de, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:14.701337 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd6fff-423f-4d3b-8bf8-2aaa3a1879a9 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:14.771819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 913/1796] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:14 2022] PUT /identity/v3/projects/730a8bf0156545a2b8a0d926002bf809/users/92175ee3c02d4c919bfdbb26d5e4a4de/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:14.782999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4caed5-e0dd-43bd-a6ed-e2f2fbbae01a None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:14.783282 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4caed5-e0dd-43bd-a6ed-e2f2fbbae01a None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:14.783555 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4caed5-e0dd-43bd-a6ed-e2f2fbbae01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:14.813824 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-3b4caed5-e0dd-43bd-a6ed-e2f2fbbae01a None None] MFA Rules not processed for user `92175ee3c02d4c919bfdbb26d5e4a4de`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:14.815067 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3b4caed5-e0dd-43bd-a6ed-e2f2fbbae01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:14.884205 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 884/1797] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:14.888794 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd6fff-423f-4d3b-8bf8-2aaa3a1879a9 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:14.951389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd6fff-423f-4d3b-8bf8-2aaa3a1879a9 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:14.956325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577277e-8e0b-4eed-a44e-2a979c9ef4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:14.957037 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577277e-8e0b-4eed-a44e-2a979c9ef4dc admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:14.957319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577277e-8e0b-4eed-a44e-2a979c9ef4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:14.957589 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577277e-8e0b-4eed-a44e-2a979c9ef4dc admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:14.958289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d577277e-8e0b-4eed-a44e-2a979c9ef4dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:14.961251 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d577277e-8e0b-4eed-a44e-2a979c9ef4dc admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:15.070669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 914/1798] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:44:14 2022] POST /identity/v3/projects => generated 355 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:15.074964 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4caed5-e0dd-43bd-a6ed-e2f2fbbae01a None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:15.087800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4caed5-e0dd-43bd-a6ed-e2f2fbbae01a None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:15.166264 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8a85a-4474-418a-a2d1-b87ea6461b86 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:15.166878 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8a85a-4474-418a-a2d1-b87ea6461b86 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:15.167367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8a85a-4474-418a-a2d1-b87ea6461b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:15.203586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8f9c2-1a5e-42d2-9e63-7f37f39dc6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:15.204197 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8f9c2-1a5e-42d2-9e63-7f37f39dc6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:15.204447 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8f9c2-1a5e-42d2-9e63-7f37f39dc6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:15.204718 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8f9c2-1a5e-42d2-9e63-7f37f39dc6e2 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:15.205423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8f9c2-1a5e-42d2-9e63-7f37f39dc6e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:15.207536 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8f9c2-1a5e-42d2-9e63-7f37f39dc6e2 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:15.217957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-7ff8a85a-4474-418a-a2d1-b87ea6461b86 None None] MFA Rules not processed for user `cb802d01e71b486cb40c6388430d674d`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:15.221904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-7ff8a85a-4474-418a-a2d1-b87ea6461b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:15.301717 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 915/1799] 10.222.0.40 () {58 vars in 1015 bytes} [Mon Aug 29 22:44:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:15.327407 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:15.342942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 885/1800] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:15 2022] POST /identity/v3/users => generated 284 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:15.347365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8a85a-4474-418a-a2d1-b87ea6461b86 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:15.367777 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8a85a-4474-418a-a2d1-b87ea6461b86 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:15.373440 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40c60c-0299-4a65-a621-b4408dea6322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:15.374358 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40c60c-0299-4a65-a621-b4408dea6322 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:15.374710 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40c60c-0299-4a65-a621-b4408dea6322 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:15.375057 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40c60c-0299-4a65-a621-b4408dea6322 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:15.375836 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40c60c-0299-4a65-a621-b4408dea6322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:15.380457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40c60c-0299-4a65-a621-b4408dea6322 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:15.385380 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8f9c2-1a5e-42d2-9e63-7f37f39dc6e2 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:15.388658 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-feb8f9c2-1a5e-42d2-9e63-7f37f39dc6e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:15.394721 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 916/1801] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:15.399640 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40c60c-0299-4a65-a621-b4408dea6322 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:15.417245 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8f9c2-1a5e-42d2-9e63-7f37f39dc6e2 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:15.418596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40c60c-0299-4a65-a621-b4408dea6322 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:15.426121 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:15.433465 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d6d44-2aa6-4504-b30e-b3e2640ca5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:15.435108 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d6d44-2aa6-4504-b30e-b3e2640ca5f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:15.435863 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d6d44-2aa6-4504-b30e-b3e2640ca5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:15.436359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d6d44-2aa6-4504-b30e-b3e2640ca5f7 admin admin] PATH_INFO: `/v3/projects/6518fcb59dd54cf7b3fc9a6558377b32/users/2cba78a0b00446eb99cf55ab1f42ebba/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:15.441956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd07f31-a86b-4b4a-9ee6-d06d2a71c821 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:15.442461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd07f31-a86b-4b4a-9ee6-d06d2a71c821 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:15.443420 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd07f31-a86b-4b4a-9ee6-d06d2a71c821 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:15.444243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd07f31-a86b-4b4a-9ee6-d06d2a71c821 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:15.458024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d6d44-2aa6-4504-b30e-b3e2640ca5f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6518fcb59dd54cf7b3fc9a6558377b32, user_id=2cba78a0b00446eb99cf55ab1f42ebba, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:15.462611 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d6d44-2aa6-4504-b30e-b3e2640ca5f7 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:15.469955 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd07f31-a86b-4b4a-9ee6-d06d2a71c821 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:15.472716 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd07f31-a86b-4b4a-9ee6-d06d2a71c821 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:15.487341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 886/1802] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:15.562406 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 917/1803] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:15 2022] PUT /identity/v3/projects/6518fcb59dd54cf7b3fc9a6558377b32/users/2cba78a0b00446eb99cf55ab1f42ebba/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:15.564221 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd07f31-a86b-4b4a-9ee6-d06d2a71c821 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:15.586947 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd07f31-a86b-4b4a-9ee6-d06d2a71c821 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:15.729249 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5edf5f-84ac-470f-b294-48c59379b832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:15.730177 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5edf5f-84ac-470f-b294-48c59379b832 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:15.730634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5edf5f-84ac-470f-b294-48c59379b832 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:15.731076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5edf5f-84ac-470f-b294-48c59379b832 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:15.732340 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5edf5f-84ac-470f-b294-48c59379b832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:15.734208 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5edf5f-84ac-470f-b294-48c59379b832 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:15.777611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 887/1804] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:15.792417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d6d44-2aa6-4504-b30e-b3e2640ca5f7 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:15.827693 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d6d44-2aa6-4504-b30e-b3e2640ca5f7 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:15.838159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae23bcb-403d-449e-9893-2d4aa611b623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:15.839124 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae23bcb-403d-449e-9893-2d4aa611b623 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:15.839545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae23bcb-403d-449e-9893-2d4aa611b623 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:15.839952 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae23bcb-403d-449e-9893-2d4aa611b623 admin admin] PATH_INFO: `/v3/projects/6518fcb59dd54cf7b3fc9a6558377b32/users/2cba78a0b00446eb99cf55ab1f42ebba/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:15.842491 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae23bcb-403d-449e-9893-2d4aa611b623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6518fcb59dd54cf7b3fc9a6558377b32, user_id=2cba78a0b00446eb99cf55ab1f42ebba, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:15.845308 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae23bcb-403d-449e-9893-2d4aa611b623 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:15.858904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 918/1805] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:15 2022] PUT /identity/v3/projects/6518fcb59dd54cf7b3fc9a6558377b32/users/2cba78a0b00446eb99cf55ab1f42ebba/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:15.874182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c62f69-d60b-44fd-9c12-25f4711608e2 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:15.874692 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c62f69-d60b-44fd-9c12-25f4711608e2 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:15.875050 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c62f69-d60b-44fd-9c12-25f4711608e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:15.908123 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae23bcb-403d-449e-9893-2d4aa611b623 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:15.910998 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-fae23bcb-403d-449e-9893-2d4aa611b623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:16.014077 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae23bcb-403d-449e-9893-2d4aa611b623 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:16.021552 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b89fb4-7d36-4944-8da7-e61c3a67474c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:16.022877 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b89fb4-7d36-4944-8da7-e61c3a67474c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:16.023332 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b89fb4-7d36-4944-8da7-e61c3a67474c admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:16.023869 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b89fb4-7d36-4944-8da7-e61c3a67474c admin admin] PATH_INFO: `/v3/users/2592a7be71ec4f35bb2a2b5fba8dab56` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:16.025731 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b89fb4-7d36-4944-8da7-e61c3a67474c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2592a7be71ec4f35bb2a2b5fba8dab56)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:16.025966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-37c62f69-d60b-44fd-9c12-25f4711608e2 None None] MFA Rules not processed for user `2cba78a0b00446eb99cf55ab1f42ebba`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:16.027230 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-37c62f69-d60b-44fd-9c12-25f4711608e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:16.028396 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b89fb4-7d36-4944-8da7-e61c3a67474c admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:16.206590 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 888/1806] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:16.206590 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b8e0a-ec9b-47f1-9868-0d6a2f2decda None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:16.206590 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b8e0a-ec9b-47f1-9868-0d6a2f2decda None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:16.206590 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b8e0a-ec9b-47f1-9868-0d6a2f2decda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:16.273063 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-a46b8e0a-ec9b-47f1-9868-0d6a2f2decda None None] MFA Rules not processed for user `980157efae8f43b59dc063fda04c0348`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:16.274240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a46b8e0a-ec9b-47f1-9868-0d6a2f2decda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:16.324994 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-81b89fb4-7d36-4944-8da7-e61c3a67474c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2592a7be71ec4f35bb2a2b5fba8dab56'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:16.326304 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-81b89fb4-7d36-4944-8da7-e61c3a67474c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2592a7be71ec4f35bb2a2b5fba8dab56'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:16.342308 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-81b89fb4-7d36-4944-8da7-e61c3a67474c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2592a7be71ec4f35bb2a2b5fba8dab56'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:16.370106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 889/1807] 10.222.0.59 () {58 vars in 1009 bytes} [Mon Aug 29 22:44:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:16.456807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-81b89fb4-7d36-4944-8da7-e61c3a67474c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2592a7be71ec4f35bb2a2b5fba8dab56'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:16.457445 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-81b89fb4-7d36-4944-8da7-e61c3a67474c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2592a7be71ec4f35bb2a2b5fba8dab56'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:16.505646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 919/1808] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:44:15 2022] DELETE /identity/v3/users/2592a7be71ec4f35bb2a2b5fba8dab56 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:16.508103 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05dd8b-92a9-47d5-b941-2134104b3c5d None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:16.508769 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05dd8b-92a9-47d5-b941-2134104b3c5d None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:16.509244 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05dd8b-92a9-47d5-b941-2134104b3c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:16.535967 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-6a05dd8b-92a9-47d5-b941-2134104b3c5d None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:16.538285 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6a05dd8b-92a9-47d5-b941-2134104b3c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:16.609291 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 890/1809] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:16.618254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b89fb4-7d36-4944-8da7-e61c3a67474c admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:16.619687 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-81b89fb4-7d36-4944-8da7-e61c3a67474c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:16.623249 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a05dd8b-92a9-47d5-b941-2134104b3c5d None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:16.625289 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6a05dd8b-92a9-47d5-b941-2134104b3c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:16.701067 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a05dd8b-92a9-47d5-b941-2134104b3c5d None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:16.705013 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366cda38-1fae-4808-8feb-f832a783a68f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:16.705734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cda38-1fae-4808-8feb-f832a783a68f service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:16.706168 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cda38-1fae-4808-8feb-f832a783a68f service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:16.706964 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cda38-1fae-4808-8feb-f832a783a68f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:16.720611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366cda38-1fae-4808-8feb-f832a783a68f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:16.722525 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366cda38-1fae-4808-8feb-f832a783a68f service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:16.733567 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b89fb4-7d36-4944-8da7-e61c3a67474c admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:16.737538 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 891/1810] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:44:16.739249 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76725d74-8fc5-4b3e-87c9-bd710ef9fd87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:16.741487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76725d74-8fc5-4b3e-87c9-bd710ef9fd87 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:16.741774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76725d74-8fc5-4b3e-87c9-bd710ef9fd87 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:16.742105 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76725d74-8fc5-4b3e-87c9-bd710ef9fd87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:16.745106 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-76725d74-8fc5-4b3e-87c9-bd710ef9fd87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:16.839559 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366cda38-1fae-4808-8feb-f832a783a68f service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:16.843861 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-366cda38-1fae-4808-8feb-f832a783a68f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:16.880643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76725d74-8fc5-4b3e-87c9-bd710ef9fd87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:16.883396 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76725d74-8fc5-4b3e-87c9-bd710ef9fd87 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:16.910548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 920/1811] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:16 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:17.005261 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366cda38-1fae-4808-8feb-f832a783a68f service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:17.016218 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133c4f2-c11e-4c79-b7ca-f21e1d8fed5f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f0503ae1b24ce6a8adb3cfd7973f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:17.017927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133c4f2-c11e-4c79-b7ca-f21e1d8fed5f service swift] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:17.018210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133c4f2-c11e-4c79-b7ca-f21e1d8fed5f service swift] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:17.018478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133c4f2-c11e-4c79-b7ca-f21e1d8fed5f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:17.110199 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133c4f2-c11e-4c79-b7ca-f21e1d8fed5f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:17.112786 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133c4f2-c11e-4c79-b7ca-f21e1d8fed5f service swift] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:17.124823 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 892/1812] 10.222.0.59 () {60 vars in 1384 bytes} [Mon Aug 29 22:44:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 22:44:17.181454 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76725d74-8fc5-4b3e-87c9-bd710ef9fd87 service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:17.196087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76725d74-8fc5-4b3e-87c9-bd710ef9fd87 service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:17.203001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79ea26-dfee-4305-899b-f4e8723062bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:17.204858 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79ea26-dfee-4305-899b-f4e8723062bd service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:17.205365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79ea26-dfee-4305-899b-f4e8723062bd service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:17.205782 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79ea26-dfee-4305-899b-f4e8723062bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:17.208534 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3a79ea26-dfee-4305-899b-f4e8723062bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:17.351312 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79ea26-dfee-4305-899b-f4e8723062bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:17.356378 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79ea26-dfee-4305-899b-f4e8723062bd service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:17.387403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 921/1813] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:17 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:17.418559 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:18.916908 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133c4f2-c11e-4c79-b7ca-f21e1d8fed5f service swift] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:18.921008 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-e133c4f2-c11e-4c79-b7ca-f21e1d8fed5f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:18.955118 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133c4f2-c11e-4c79-b7ca-f21e1d8fed5f service swift] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:18.961494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc3cc4-a3c0-4c60-a5a7-8948928a4e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:18.962193 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc3cc4-a3c0-4c60-a5a7-8948928a4e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:18.962486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc3cc4-a3c0-4c60-a5a7-8948928a4e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:18.962751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc3cc4-a3c0-4c60-a5a7-8948928a4e39 admin admin] PATH_INFO: `/v3/projects/623fb48adc4147f8ab2c7037278eb2b5` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:18.963899 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc3cc4-a3c0-4c60-a5a7-8948928a4e39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=623fb48adc4147f8ab2c7037278eb2b5)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:18.965883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc3cc4-a3c0-4c60-a5a7-8948928a4e39 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:19.095259 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758887d-32af-4dad-a87c-2583af3f8253 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:19.095865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758887d-32af-4dad-a87c-2583af3f8253 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:19.096155 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758887d-32af-4dad-a87c-2583af3f8253 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:19.098065 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 922/1814] 10.222.0.59 () {54 vars in 921 bytes} [Mon Aug 29 22:44:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:44:19.136453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022362ad-7a1e-476f-80bc-90aae832c7f5 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:19.137025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022362ad-7a1e-476f-80bc-90aae832c7f5 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:19.137703 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022362ad-7a1e-476f-80bc-90aae832c7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:19.190549 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-022362ad-7a1e-476f-80bc-90aae832c7f5 None None] MFA Rules not processed for user `7f21105679f245a19ea4850e6c010dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:19.191980 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-022362ad-7a1e-476f-80bc-90aae832c7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:19.262391 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-88cc3cc4-a3c0-4c60-a5a7-8948928a4e39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '623fb48adc4147f8ab2c7037278eb2b5', 'request_id': 'req-88cc3cc4-a3c0-4c60-a5a7-8948928a4e39'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:19.297269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:19.377516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-88cc3cc4-a3c0-4c60-a5a7-8948928a4e39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '623fb48adc4147f8ab2c7037278eb2b5', 'request_id': 'req-88cc3cc4-a3c0-4c60-a5a7-8948928a4e39'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:19.394724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-88cc3cc4-a3c0-4c60-a5a7-8948928a4e39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '623fb48adc4147f8ab2c7037278eb2b5', 'request_id': 'req-88cc3cc4-a3c0-4c60-a5a7-8948928a4e39'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:19.418282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 923/1815] 10.222.0.59 () {58 vars in 1018 bytes} [Mon Aug 29 22:44:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:44:19.422952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-88cc3cc4-a3c0-4c60-a5a7-8948928a4e39 admin admin] The token cache is being invalidate because project 623fb48adc4147f8ab2c7037278eb2b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:44:19.423517 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-88cc3cc4-a3c0-4c60-a5a7-8948928a4e39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:19.425464 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-88cc3cc4-a3c0-4c60-a5a7-8948928a4e39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:19.430084 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 893/1816] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:44:18 2022] DELETE /identity/v3/projects/623fb48adc4147f8ab2c7037278eb2b5 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:19.434635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022362ad-7a1e-476f-80bc-90aae832c7f5 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:19.471898 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc3cc4-a3c0-4c60-a5a7-8948928a4e39 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:19.477769 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-88cc3cc4-a3c0-4c60-a5a7-8948928a4e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:19.479318 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022362ad-7a1e-476f-80bc-90aae832c7f5 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:19.668265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b75ab-d2f5-4fa6-ab6c-db8e59c31cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:19.669101 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b75ab-d2f5-4fa6-ab6c-db8e59c31cd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:19.669440 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b75ab-d2f5-4fa6-ab6c-db8e59c31cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:19.669724 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b75ab-d2f5-4fa6-ab6c-db8e59c31cd6 admin admin] PATH_INFO: `/v3/users/dfcc874269b8406799d07c4b2fd717fa` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:19.671472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b75ab-d2f5-4fa6-ab6c-db8e59c31cd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfcc874269b8406799d07c4b2fd717fa)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:19.689344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b75ab-d2f5-4fa6-ab6c-db8e59c31cd6 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:19.861018 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc3cc4-a3c0-4c60-a5a7-8948928a4e39 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:19.871469 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95dd9de-ee88-4d4d-b3b8-0ce306d46b19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:19.875705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95dd9de-ee88-4d4d-b3b8-0ce306d46b19 service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:19.876039 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95dd9de-ee88-4d4d-b3b8-0ce306d46b19 service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:19.876364 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95dd9de-ee88-4d4d-b3b8-0ce306d46b19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:19.879577 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d95dd9de-ee88-4d4d-b3b8-0ce306d46b19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:19.919528 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95dd9de-ee88-4d4d-b3b8-0ce306d46b19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:19.921751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95dd9de-ee88-4d4d-b3b8-0ce306d46b19 service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:19.989436 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 894/1817] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:20.058605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7a2b75ab-d2f5-4fa6-ab6c-db8e59c31cd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfcc874269b8406799d07c4b2fd717fa'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:20.063880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7a2b75ab-d2f5-4fa6-ab6c-db8e59c31cd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfcc874269b8406799d07c4b2fd717fa'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:20.102354 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7a2b75ab-d2f5-4fa6-ab6c-db8e59c31cd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfcc874269b8406799d07c4b2fd717fa'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:20.133989 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:20.212075 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7a2b75ab-d2f5-4fa6-ab6c-db8e59c31cd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfcc874269b8406799d07c4b2fd717fa'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:20.212827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7a2b75ab-d2f5-4fa6-ab6c-db8e59c31cd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfcc874269b8406799d07c4b2fd717fa'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:20.250402 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 924/1818] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:44:19 2022] DELETE /identity/v3/users/dfcc874269b8406799d07c4b2fd717fa => generated 0 bytes in 816 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:20.564094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb5b36-b0c6-4386-ac24-9e96ce98e36f None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:20.564952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb5b36-b0c6-4386-ac24-9e96ce98e36f None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:20.565588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb5b36-b0c6-4386-ac24-9e96ce98e36f None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:20.567368 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 895/1819] 10.222.0.59 () {54 vars in 921 bytes} [Mon Aug 29 22:44:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:44:20.592842 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e12fdd-6354-4ce2-8c0d-466a7b25e352 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:20.593160 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e12fdd-6354-4ce2-8c0d-466a7b25e352 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:20.593437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e12fdd-6354-4ce2-8c0d-466a7b25e352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:20.651083 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-46e12fdd-6354-4ce2-8c0d-466a7b25e352 None None] MFA Rules not processed for user `7f21105679f245a19ea4850e6c010dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:20.674210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-46e12fdd-6354-4ce2-8c0d-466a7b25e352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:20.887778 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 925/1820] 10.222.0.59 () {58 vars in 1018 bytes} [Mon Aug 29 22:44:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:21.304890 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb5b36-b0c6-4386-ac24-9e96ce98e36f None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:21.306780 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-35fb5b36-b0c6-4386-ac24-9e96ce98e36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:21.346790 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb5b36-b0c6-4386-ac24-9e96ce98e36f None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:21.365592 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dba26b-2628-4975-bdef-d47a990d3e56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:21.366628 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dba26b-2628-4975-bdef-d47a990d3e56 service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:21.367075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dba26b-2628-4975-bdef-d47a990d3e56 service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:21.367495 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dba26b-2628-4975-bdef-d47a990d3e56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:21.381971 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dba26b-2628-4975-bdef-d47a990d3e56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:21.383842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dba26b-2628-4975-bdef-d47a990d3e56 service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:21.405666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 896/1821] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:21.563172 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e12fdd-6354-4ce2-8c0d-466a7b25e352 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:21.565235 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-46e12fdd-6354-4ce2-8c0d-466a7b25e352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:21.663626 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:21.691967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e12fdd-6354-4ce2-8c0d-466a7b25e352 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:21.700932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046af2b-3f75-4baf-b645-3adb20497e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:21.705364 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046af2b-3f75-4baf-b645-3adb20497e3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:21.705364 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046af2b-3f75-4baf-b645-3adb20497e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:21.705364 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046af2b-3f75-4baf-b645-3adb20497e3c admin admin] PATH_INFO: `/v3/users/a40a9d0e8e6d4b15becb654edeeba0b0` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:21.705364 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046af2b-3f75-4baf-b645-3adb20497e3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a40a9d0e8e6d4b15becb654edeeba0b0)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:21.707320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046af2b-3f75-4baf-b645-3adb20497e3c admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:22.010891 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-a046af2b-3f75-4baf-b645-3adb20497e3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a40a9d0e8e6d4b15becb654edeeba0b0'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:22.012896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-a046af2b-3f75-4baf-b645-3adb20497e3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a40a9d0e8e6d4b15becb654edeeba0b0'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:22.054546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-a046af2b-3f75-4baf-b645-3adb20497e3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a40a9d0e8e6d4b15becb654edeeba0b0'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:22.205845 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-a046af2b-3f75-4baf-b645-3adb20497e3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a40a9d0e8e6d4b15becb654edeeba0b0'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:22.206918 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-a046af2b-3f75-4baf-b645-3adb20497e3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a40a9d0e8e6d4b15becb654edeeba0b0'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:22.269713 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 926/1822] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:44:21 2022] DELETE /identity/v3/users/a40a9d0e8e6d4b15becb654edeeba0b0 => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:22.288302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2208d-610f-4d8c-be04-96bd9e921051 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:22.288764 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2208d-610f-4d8c-be04-96bd9e921051 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:22.289131 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2208d-610f-4d8c-be04-96bd9e921051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:22.317240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-4cf2208d-610f-4d8c-be04-96bd9e921051 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:22.321366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-4cf2208d-610f-4d8c-be04-96bd9e921051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:22.430770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 897/1823] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:22.471197 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046af2b-3f75-4baf-b645-3adb20497e3c admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:22.473010 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a046af2b-3f75-4baf-b645-3adb20497e3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:22.579017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046af2b-3f75-4baf-b645-3adb20497e3c admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:22.593076 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069587b-6f36-47f7-8c4a-51e84a5ef5e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:22.593076 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069587b-6f36-47f7-8c4a-51e84a5ef5e2 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:22.593076 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069587b-6f36-47f7-8c4a-51e84a5ef5e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:22.593076 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069587b-6f36-47f7-8c4a-51e84a5ef5e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:22.618953 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069587b-6f36-47f7-8c4a-51e84a5ef5e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:22.621598 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069587b-6f36-47f7-8c4a-51e84a5ef5e2 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:22.658267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 927/1824] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:22.852385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:23.249854 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2208d-610f-4d8c-be04-96bd9e921051 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:23.253077 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-4cf2208d-610f-4d8c-be04-96bd9e921051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:23.299169 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2208d-610f-4d8c-be04-96bd9e921051 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:23.315036 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305161c4-78f1-4eb1-b2f0-5d7f3f797c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:23.316927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305161c4-78f1-4eb1-b2f0-5d7f3f797c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:23.317468 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305161c4-78f1-4eb1-b2f0-5d7f3f797c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:23.319042 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305161c4-78f1-4eb1-b2f0-5d7f3f797c97 admin admin] PATH_INFO: `/v3/projects/b348bd4ebe5b49cbbb393e862a8389ab` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:23.320593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305161c4-78f1-4eb1-b2f0-5d7f3f797c97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b348bd4ebe5b49cbbb393e862a8389ab)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:23.323722 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305161c4-78f1-4eb1-b2f0-5d7f3f797c97 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:23.609900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-305161c4-78f1-4eb1-b2f0-5d7f3f797c97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b348bd4ebe5b49cbbb393e862a8389ab', 'request_id': 'req-305161c4-78f1-4eb1-b2f0-5d7f3f797c97'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:23.721122 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-305161c4-78f1-4eb1-b2f0-5d7f3f797c97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b348bd4ebe5b49cbbb393e862a8389ab', 'request_id': 'req-305161c4-78f1-4eb1-b2f0-5d7f3f797c97'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:23.780667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-305161c4-78f1-4eb1-b2f0-5d7f3f797c97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b348bd4ebe5b49cbbb393e862a8389ab', 'request_id': 'req-305161c4-78f1-4eb1-b2f0-5d7f3f797c97'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:23.822311 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-305161c4-78f1-4eb1-b2f0-5d7f3f797c97 admin admin] The token cache is being invalidate because project b348bd4ebe5b49cbbb393e862a8389ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:44:23.822773 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-305161c4-78f1-4eb1-b2f0-5d7f3f797c97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:23.823997 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-305161c4-78f1-4eb1-b2f0-5d7f3f797c97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:23.826377 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 898/1825] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:44:23 2022] DELETE /identity/v3/projects/b348bd4ebe5b49cbbb393e862a8389ab => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:44:23.832103 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3f7ad-09ed-462a-8beb-b167d83dd84b None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:23.832713 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3f7ad-09ed-462a-8beb-b167d83dd84b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:23.832818 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3f7ad-09ed-462a-8beb-b167d83dd84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:23.901900 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-6ff3f7ad-09ed-462a-8beb-b167d83dd84b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:23.905744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6ff3f7ad-09ed-462a-8beb-b167d83dd84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:23.958300 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305161c4-78f1-4eb1-b2f0-5d7f3f797c97 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:23.960292 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-305161c4-78f1-4eb1-b2f0-5d7f3f797c97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:24.016532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 928/1826] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:24.024865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929d1d4-b980-494c-8f24-393e366ab038 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:24.025370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929d1d4-b980-494c-8f24-393e366ab038 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:24.025763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929d1d4-b980-494c-8f24-393e366ab038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:24.088450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-d929d1d4-b980-494c-8f24-393e366ab038 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:24.093672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-d929d1d4-b980-494c-8f24-393e366ab038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:24.117583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 929/1827] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:24.121958 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929d1d4-b980-494c-8f24-393e366ab038 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:24.136555 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929d1d4-b980-494c-8f24-393e366ab038 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:24.144182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305161c4-78f1-4eb1-b2f0-5d7f3f797c97 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:24.151082 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffaf31-3cb4-43b9-911e-a1460897dd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:24.151669 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7bbd1-e2d5-4b06-84f9-daf1208cc4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:24.151975 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffaf31-3cb4-43b9-911e-a1460897dd05 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:24.153012 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7bbd1-e2d5-4b06-84f9-daf1208cc4e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:24.153515 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7bbd1-e2d5-4b06-84f9-daf1208cc4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:24.153634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffaf31-3cb4-43b9-911e-a1460897dd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:24.153841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffaf31-3cb4-43b9-911e-a1460897dd05 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:24.154286 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7bbd1-e2d5-4b06-84f9-daf1208cc4e5 admin admin] PATH_INFO: `/v3/projects/53e6c32266fb4fbe96a125c5d2a81812` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:24.155592 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffaf31-3cb4-43b9-911e-a1460897dd05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:44:24.155895 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffaf31-3cb4-43b9-911e-a1460897dd05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:24.156086 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7bbd1-e2d5-4b06-84f9-daf1208cc4e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53e6c32266fb4fbe96a125c5d2a81812)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:24.159179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffaf31-3cb4-43b9-911e-a1460897dd05 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:24.159441 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7bbd1-e2d5-4b06-84f9-daf1208cc4e5 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:24.206628 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 930/1828] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:44:24.214228 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffaf31-3cb4-43b9-911e-a1460897dd05 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:24.218027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e4ffaf31-3cb4-43b9-911e-a1460897dd05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:24.255045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffaf31-3cb4-43b9-911e-a1460897dd05 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:24.262346 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79d15e-77d1-4694-a086-5cff2503e0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:24.263942 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79d15e-77d1-4694-a086-5cff2503e0d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:24.264595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79d15e-77d1-4694-a086-5cff2503e0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:24.265070 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79d15e-77d1-4694-a086-5cff2503e0d3 admin admin] PATH_INFO: `/v3/users/4267fbbf0c01465c99320c848e71fda0` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:24.267718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79d15e-77d1-4694-a086-5cff2503e0d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4267fbbf0c01465c99320c848e71fda0)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:24.270049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79d15e-77d1-4694-a086-5cff2503e0d3 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:24.345102 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2fd7bbd1-e2d5-4b06-84f9-daf1208cc4e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53e6c32266fb4fbe96a125c5d2a81812', 'request_id': 'req-2fd7bbd1-e2d5-4b06-84f9-daf1208cc4e5'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:24.462425 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2fd7bbd1-e2d5-4b06-84f9-daf1208cc4e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53e6c32266fb4fbe96a125c5d2a81812', 'request_id': 'req-2fd7bbd1-e2d5-4b06-84f9-daf1208cc4e5'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:24.470749 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2fd7bbd1-e2d5-4b06-84f9-daf1208cc4e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53e6c32266fb4fbe96a125c5d2a81812', 'request_id': 'req-2fd7bbd1-e2d5-4b06-84f9-daf1208cc4e5'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:24.486889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2fd7bbd1-e2d5-4b06-84f9-daf1208cc4e5 admin admin] The token cache is being invalidate because project 53e6c32266fb4fbe96a125c5d2a81812 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:44:24.487730 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2fd7bbd1-e2d5-4b06-84f9-daf1208cc4e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:24.490124 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2fd7bbd1-e2d5-4b06-84f9-daf1208cc4e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:24.495181 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 899/1829] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:44:23 2022] DELETE /identity/v3/projects/53e6c32266fb4fbe96a125c5d2a81812 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:24.498139 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7bbd1-e2d5-4b06-84f9-daf1208cc4e5 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:24.500033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-2fd7bbd1-e2d5-4b06-84f9-daf1208cc4e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:24.622812 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6d79d15e-77d1-4694-a086-5cff2503e0d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4267fbbf0c01465c99320c848e71fda0'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:24.628478 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6d79d15e-77d1-4694-a086-5cff2503e0d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4267fbbf0c01465c99320c848e71fda0'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:24.658396 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6d79d15e-77d1-4694-a086-5cff2503e0d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4267fbbf0c01465c99320c848e71fda0'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:24.670453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7bbd1-e2d5-4b06-84f9-daf1208cc4e5 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:24.685248 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdeb43-c1a0-4934-9897-2c8b04f936d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:24.686219 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdeb43-c1a0-4934-9897-2c8b04f936d2 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:24.686706 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdeb43-c1a0-4934-9897-2c8b04f936d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:24.687180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdeb43-c1a0-4934-9897-2c8b04f936d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:24.688042 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdeb43-c1a0-4934-9897-2c8b04f936d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:24.690874 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdeb43-c1a0-4934-9897-2c8b04f936d2 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:24.748920 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6d79d15e-77d1-4694-a086-5cff2503e0d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4267fbbf0c01465c99320c848e71fda0'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:24.749590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6d79d15e-77d1-4694-a086-5cff2503e0d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4267fbbf0c01465c99320c848e71fda0'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:24.781318 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 900/1830] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:44:24 2022] POST /identity/v3/projects => generated 375 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:24.782633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdeb43-c1a0-4934-9897-2c8b04f936d2 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:24.784977 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-7ccdeb43-c1a0-4934-9897-2c8b04f936d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:24.798483 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 931/1831] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:44:24 2022] DELETE /identity/v3/users/4267fbbf0c01465c99320c848e71fda0 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:24.798967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79d15e-77d1-4694-a086-5cff2503e0d3 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:24.806493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6d79d15e-77d1-4694-a086-5cff2503e0d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:24.883628 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdeb43-c1a0-4934-9897-2c8b04f936d2 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:24.928356 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:24.955006 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:24.987872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662a637-0160-4ffc-a7b0-1287a5157ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:24.993077 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662a637-0160-4ffc-a7b0-1287a5157ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:24.993077 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662a637-0160-4ffc-a7b0-1287a5157ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:24.993077 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662a637-0160-4ffc-a7b0-1287a5157ba0 admin admin] PATH_INFO: `/v3/users/2ee5c508f1d144f186e0cfa65e736712` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:24.993377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79d15e-77d1-4694-a086-5cff2503e0d3 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:24.993465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662a637-0160-4ffc-a7b0-1287a5157ba0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ee5c508f1d144f186e0cfa65e736712)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:24.993548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662a637-0160-4ffc-a7b0-1287a5157ba0 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:25.012859 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba21570-18ed-486e-8f8d-e616372ce2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:25.015994 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba21570-18ed-486e-8f8d-e616372ce2ee admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:25.015994 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba21570-18ed-486e-8f8d-e616372ce2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:25.015994 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba21570-18ed-486e-8f8d-e616372ce2ee admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:25.015994 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba21570-18ed-486e-8f8d-e616372ce2ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:25.022861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba21570-18ed-486e-8f8d-e616372ce2ee admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:25.148559 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 932/1832] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:24 2022] POST /identity/v3/users => generated 288 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:25.148559 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba6561-a905-48c4-b23a-ce7898b96450 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:25.148559 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba6561-a905-48c4-b23a-ce7898b96450 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:25.148559 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba6561-a905-48c4-b23a-ce7898b96450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:25.176323 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-69ba6561-a905-48c4-b23a-ce7898b96450 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:25.178893 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-69ba6561-a905-48c4-b23a-ce7898b96450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:25.189510 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 933/1833] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:25.191937 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba6561-a905-48c4-b23a-ce7898b96450 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:25.202801 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba6561-a905-48c4-b23a-ce7898b96450 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:25.204666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-3662a637-0160-4ffc-a7b0-1287a5157ba0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ee5c508f1d144f186e0cfa65e736712'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:25.210775 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b01951-ac38-4d9b-83f1-9842cfbc744a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:25.211739 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b01951-ac38-4d9b-83f1-9842cfbc744a admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:25.212292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b01951-ac38-4d9b-83f1-9842cfbc744a admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:25.212720 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b01951-ac38-4d9b-83f1-9842cfbc744a admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:25.213622 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b01951-ac38-4d9b-83f1-9842cfbc744a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:25.215854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b01951-ac38-4d9b-83f1-9842cfbc744a admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:25.224285 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-3662a637-0160-4ffc-a7b0-1287a5157ba0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ee5c508f1d144f186e0cfa65e736712'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:25.226889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-3662a637-0160-4ffc-a7b0-1287a5157ba0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ee5c508f1d144f186e0cfa65e736712'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:25.231970 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 934/1834] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:25.231970 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b01951-ac38-4d9b-83f1-9842cfbc744a admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:25.234166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-50b01951-ac38-4d9b-83f1-9842cfbc744a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:25.249682 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-3662a637-0160-4ffc-a7b0-1287a5157ba0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ee5c508f1d144f186e0cfa65e736712'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:25.250738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-3662a637-0160-4ffc-a7b0-1287a5157ba0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ee5c508f1d144f186e0cfa65e736712'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:25.305896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b01951-ac38-4d9b-83f1-9842cfbc744a admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:25.310109 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4d7e3-8735-41e2-b355-db2fcbb195a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:25.310719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4d7e3-8735-41e2-b355-db2fcbb195a3 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:25.311043 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4d7e3-8735-41e2-b355-db2fcbb195a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:25.311318 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4d7e3-8735-41e2-b355-db2fcbb195a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:25.313407 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-ece4d7e3-8735-41e2-b355-db2fcbb195a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:25.330294 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4d7e3-8735-41e2-b355-db2fcbb195a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:25.331671 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4d7e3-8735-41e2-b355-db2fcbb195a3 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:25.342661 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 901/1835] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:44:24 2022] DELETE /identity/v3/users/2ee5c508f1d144f186e0cfa65e736712 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:25.343510 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662a637-0160-4ffc-a7b0-1287a5157ba0 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:25.345015 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3662a637-0160-4ffc-a7b0-1287a5157ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:25.346913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 935/1836] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:25.432329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662a637-0160-4ffc-a7b0-1287a5157ba0 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:25.438429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b69942-06ff-465f-a87f-86a904c2f747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:25.444589 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b69942-06ff-465f-a87f-86a904c2f747 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:25.444589 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b69942-06ff-465f-a87f-86a904c2f747 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:25.444589 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b69942-06ff-465f-a87f-86a904c2f747 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:25.444589 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b69942-06ff-465f-a87f-86a904c2f747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:25.444589 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b69942-06ff-465f-a87f-86a904c2f747 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:25.460125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 902/1837] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:25.461589 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4d7e3-8735-41e2-b355-db2fcbb195a3 service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:25.474050 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4d7e3-8735-41e2-b355-db2fcbb195a3 service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:25.486030 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2d376-8f91-4d94-a593-6fd08b814a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:25.486925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2d376-8f91-4d94-a593-6fd08b814a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:25.487287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2d376-8f91-4d94-a593-6fd08b814a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:25.487702 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2d376-8f91-4d94-a593-6fd08b814a33 admin admin] PATH_INFO: `/v3/projects/778425318e494e6a9e054b18a2f22e16/users/a201508ed31d4a6f8095e26d40e04740/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:25.514152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2d376-8f91-4d94-a593-6fd08b814a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=778425318e494e6a9e054b18a2f22e16, user_id=a201508ed31d4a6f8095e26d40e04740, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:25.518371 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2d376-8f91-4d94-a593-6fd08b814a33 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:25.641716 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 936/1838] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:25 2022] PUT /identity/v3/projects/778425318e494e6a9e054b18a2f22e16/users/a201508ed31d4a6f8095e26d40e04740/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:25.657144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa0cdd-4902-4733-8c03-a03a243df48d None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:25.657346 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa0cdd-4902-4733-8c03-a03a243df48d None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:25.657523 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa0cdd-4902-4733-8c03-a03a243df48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:25.711363 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-36fa0cdd-4902-4733-8c03-a03a243df48d None None] MFA Rules not processed for user `a201508ed31d4a6f8095e26d40e04740`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:25.712882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-36fa0cdd-4902-4733-8c03-a03a243df48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:25.842870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 903/1839] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:25.865260 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2d376-8f91-4d94-a593-6fd08b814a33 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:25.867198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8dc2d376-8f91-4d94-a593-6fd08b814a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:26.015079 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2d376-8f91-4d94-a593-6fd08b814a33 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:26.032490 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23d063-15a7-4598-9bb0-729cd0832a23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:26.033186 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23d063-15a7-4598-9bb0-729cd0832a23 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:26.033726 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23d063-15a7-4598-9bb0-729cd0832a23 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:26.034652 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23d063-15a7-4598-9bb0-729cd0832a23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:26.057873 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23d063-15a7-4598-9bb0-729cd0832a23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:26.059818 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23d063-15a7-4598-9bb0-729cd0832a23 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:26.081035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 937/1840] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:26.291328 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:26.376699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:26.623702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa0cdd-4902-4733-8c03-a03a243df48d None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:26.625196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-36fa0cdd-4902-4733-8c03-a03a243df48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:26.803610 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa0cdd-4902-4733-8c03-a03a243df48d None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:26.820257 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2ab84-2be0-499d-848e-958730cce999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:26.830496 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2ab84-2be0-499d-848e-958730cce999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:26.830778 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2ab84-2be0-499d-848e-958730cce999 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:26.831395 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2ab84-2be0-499d-848e-958730cce999 admin admin] PATH_INFO: `/v3/projects/aea5e328b0d342188f0d75a05f2edc3f` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:26.833887 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd2ab84-2be0-499d-848e-958730cce999 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aea5e328b0d342188f0d75a05f2edc3f)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:26.848588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd2ab84-2be0-499d-848e-958730cce999 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:27.070063 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-7cd2ab84-2be0-499d-848e-958730cce999 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aea5e328b0d342188f0d75a05f2edc3f', 'request_id': 'req-7cd2ab84-2be0-499d-848e-958730cce999'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:27.163087 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-7cd2ab84-2be0-499d-848e-958730cce999 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aea5e328b0d342188f0d75a05f2edc3f', 'request_id': 'req-7cd2ab84-2be0-499d-848e-958730cce999'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:27.169516 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23d063-15a7-4598-9bb0-729cd0832a23 service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:27.171279 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6e23d063-15a7-4598-9bb0-729cd0832a23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:27.185997 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-7cd2ab84-2be0-499d-848e-958730cce999 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aea5e328b0d342188f0d75a05f2edc3f', 'request_id': 'req-7cd2ab84-2be0-499d-848e-958730cce999'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:27.205354 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-7cd2ab84-2be0-499d-848e-958730cce999 admin admin] The token cache is being invalidate because project aea5e328b0d342188f0d75a05f2edc3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:44:27.205567 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-7cd2ab84-2be0-499d-848e-958730cce999 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:27.206235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-7cd2ab84-2be0-499d-848e-958730cce999 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:27.208205 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 904/1841] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:44:26 2022] DELETE /identity/v3/projects/aea5e328b0d342188f0d75a05f2edc3f => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:27.209985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23d063-15a7-4598-9bb0-729cd0832a23 service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:27.213557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2ab84-2be0-499d-848e-958730cce999 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:27.215180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-7cd2ab84-2be0-499d-848e-958730cce999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:27.316436 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84a4c0-4a2b-4026-8722-1064b4c4d89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:27.317253 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84a4c0-4a2b-4026-8722-1064b4c4d89a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:27.317908 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84a4c0-4a2b-4026-8722-1064b4c4d89a admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:27.318428 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84a4c0-4a2b-4026-8722-1064b4c4d89a admin admin] PATH_INFO: `/v3/projects/6d721dc223fe415d8aba34f44e1b9689` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:27.320042 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84a4c0-4a2b-4026-8722-1064b4c4d89a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d721dc223fe415d8aba34f44e1b9689)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:27.338751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2ab84-2be0-499d-848e-958730cce999 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:27.339064 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84a4c0-4a2b-4026-8722-1064b4c4d89a admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:27.349052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6ed66-c8ac-486e-a3e9-fbffbe6f2ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:27.349052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6ed66-c8ac-486e-a3e9-fbffbe6f2ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:27.349052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6ed66-c8ac-486e-a3e9-fbffbe6f2ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:27.349052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6ed66-c8ac-486e-a3e9-fbffbe6f2ccf admin admin] PATH_INFO: `/v3/users/ae9aee54902e48feaff7903d03aebb3e` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:27.350662 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6ed66-c8ac-486e-a3e9-fbffbe6f2ccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae9aee54902e48feaff7903d03aebb3e)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:27.353022 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6ed66-c8ac-486e-a3e9-fbffbe6f2ccf admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:27.507270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-de84a4c0-4a2b-4026-8722-1064b4c4d89a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d721dc223fe415d8aba34f44e1b9689', 'request_id': 'req-de84a4c0-4a2b-4026-8722-1064b4c4d89a'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:27.539198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-de84a4c0-4a2b-4026-8722-1064b4c4d89a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d721dc223fe415d8aba34f44e1b9689', 'request_id': 'req-de84a4c0-4a2b-4026-8722-1064b4c4d89a'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:27.547285 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-de84a4c0-4a2b-4026-8722-1064b4c4d89a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d721dc223fe415d8aba34f44e1b9689', 'request_id': 'req-de84a4c0-4a2b-4026-8722-1064b4c4d89a'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:27.605731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-69b6ed66-c8ac-486e-a3e9-fbffbe6f2ccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae9aee54902e48feaff7903d03aebb3e'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:27.614758 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-69b6ed66-c8ac-486e-a3e9-fbffbe6f2ccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae9aee54902e48feaff7903d03aebb3e'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:27.622092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-69b6ed66-c8ac-486e-a3e9-fbffbe6f2ccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae9aee54902e48feaff7903d03aebb3e'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:27.657623 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-69b6ed66-c8ac-486e-a3e9-fbffbe6f2ccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae9aee54902e48feaff7903d03aebb3e'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:27.660983 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-69b6ed66-c8ac-486e-a3e9-fbffbe6f2ccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae9aee54902e48feaff7903d03aebb3e'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:27.702389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-de84a4c0-4a2b-4026-8722-1064b4c4d89a admin admin] The token cache is being invalidate because project 6d721dc223fe415d8aba34f44e1b9689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:44:27.702389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-de84a4c0-4a2b-4026-8722-1064b4c4d89a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:27.702389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-de84a4c0-4a2b-4026-8722-1064b4c4d89a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:27.702389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 938/1842] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:44:27 2022] DELETE /identity/v3/projects/6d721dc223fe415d8aba34f44e1b9689 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:44:27.707044 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab119e82-7214-4552-beeb-9a9ce6d7200d None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:27.707210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab119e82-7214-4552-beeb-9a9ce6d7200d None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:27.707581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab119e82-7214-4552-beeb-9a9ce6d7200d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:27.748685 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-ab119e82-7214-4552-beeb-9a9ce6d7200d None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:27.750857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-ab119e82-7214-4552-beeb-9a9ce6d7200d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:27.798955 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 905/1843] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:44:27 2022] DELETE /identity/v3/users/ae9aee54902e48feaff7903d03aebb3e => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:44:27.846548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 939/1844] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:44:27.854991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc4e4b-b899-49dc-9842-15f43089afcb None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:27.855832 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc4e4b-b899-49dc-9842-15f43089afcb None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:27.856350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc4e4b-b899-49dc-9842-15f43089afcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:27.895232 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-cecc4e4b-b899-49dc-9842-15f43089afcb None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:27.901823 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-cecc4e4b-b899-49dc-9842-15f43089afcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:27.941014 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:28.077966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 906/1845] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:28.080929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:28.082291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab119e82-7214-4552-beeb-9a9ce6d7200d None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:28.101116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab119e82-7214-4552-beeb-9a9ce6d7200d None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:28.107427 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221d93c-d955-4d26-9cc8-3af8f860eee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:28.108438 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221d93c-d955-4d26-9cc8-3af8f860eee6 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:28.108861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221d93c-d955-4d26-9cc8-3af8f860eee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:28.109271 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221d93c-d955-4d26-9cc8-3af8f860eee6 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:28.109979 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221d93c-d955-4d26-9cc8-3af8f860eee6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:44:28.110391 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221d93c-d955-4d26-9cc8-3af8f860eee6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:28.112246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221d93c-d955-4d26-9cc8-3af8f860eee6 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:28.127162 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 940/1846] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:44:28.131971 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc4e4b-b899-49dc-9842-15f43089afcb None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:28.156659 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc4e4b-b899-49dc-9842-15f43089afcb None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:28.169069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a113306-4153-471d-8c7f-755bdca9a5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:28.170516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a113306-4153-471d-8c7f-755bdca9a5fe admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:28.170964 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a113306-4153-471d-8c7f-755bdca9a5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:28.171471 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a113306-4153-471d-8c7f-755bdca9a5fe admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:28.172946 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a113306-4153-471d-8c7f-755bdca9a5fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:28.177974 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a113306-4153-471d-8c7f-755bdca9a5fe admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:28.369985 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 907/1847] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:44:28 2022] POST /identity/v3/projects => generated 373 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:28.373772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221d93c-d955-4d26-9cc8-3af8f860eee6 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:28.398747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221d93c-d955-4d26-9cc8-3af8f860eee6 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:28.558210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58642da0-75f7-445b-ac09-c036d2cbd1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:28.558210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58642da0-75f7-445b-ac09-c036d2cbd1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:28.558210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58642da0-75f7-445b-ac09-c036d2cbd1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:28.558210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58642da0-75f7-445b-ac09-c036d2cbd1d3 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:28.558689 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58642da0-75f7-445b-ac09-c036d2cbd1d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:28.559624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58642da0-75f7-445b-ac09-c036d2cbd1d3 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:28.706622 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 941/1848] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:28 2022] POST /identity/v3/users => generated 287 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:28.707891 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a113306-4153-471d-8c7f-755bdca9a5fe admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:28.725439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a113306-4153-471d-8c7f-755bdca9a5fe admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:28.735498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712421f5-7b95-404a-a8d5-fa5d4f3aaccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:28.736849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712421f5-7b95-404a-a8d5-fa5d4f3aaccd admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:28.737355 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712421f5-7b95-404a-a8d5-fa5d4f3aaccd admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:28.738063 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712421f5-7b95-404a-a8d5-fa5d4f3aaccd admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:28.739324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712421f5-7b95-404a-a8d5-fa5d4f3aaccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:28.742602 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712421f5-7b95-404a-a8d5-fa5d4f3aaccd admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:28.757636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 908/1849] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:28.769188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58642da0-75f7-445b-ac09-c036d2cbd1d3 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:28.786969 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58642da0-75f7-445b-ac09-c036d2cbd1d3 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:28.796910 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c18100-0a8c-4a92-9352-8adf8ef73fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:28.797561 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c18100-0a8c-4a92-9352-8adf8ef73fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:28.797899 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c18100-0a8c-4a92-9352-8adf8ef73fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:28.799643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c18100-0a8c-4a92-9352-8adf8ef73fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:28.800291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c18100-0a8c-4a92-9352-8adf8ef73fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:28.805439 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c18100-0a8c-4a92-9352-8adf8ef73fd3 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:28.836563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 942/1850] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:28.845889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712421f5-7b95-404a-a8d5-fa5d4f3aaccd admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:28.865954 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712421f5-7b95-404a-a8d5-fa5d4f3aaccd admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:28.874150 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7152747-3cb1-476f-b8bc-ccbf8d41f4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:28.878990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7152747-3cb1-476f-b8bc-ccbf8d41f4fb admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:28.878990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7152747-3cb1-476f-b8bc-ccbf8d41f4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:28.878990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7152747-3cb1-476f-b8bc-ccbf8d41f4fb admin admin] PATH_INFO: `/v3/projects/40b5a74a98374cf999102f7cf16957d7/users/1102f5f98bf545ada39c019f45f2f413/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:28.906010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7152747-3cb1-476f-b8bc-ccbf8d41f4fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=40b5a74a98374cf999102f7cf16957d7, user_id=1102f5f98bf545ada39c019f45f2f413, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:28.909351 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7152747-3cb1-476f-b8bc-ccbf8d41f4fb admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:28.961984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c18100-0a8c-4a92-9352-8adf8ef73fd3 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:28.971416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-14c18100-0a8c-4a92-9352-8adf8ef73fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:28.982768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c18100-0a8c-4a92-9352-8adf8ef73fd3 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:29.011813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e71626-7624-4f89-8ffa-16e053feb1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:29.011813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e71626-7624-4f89-8ffa-16e053feb1eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:29.011813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e71626-7624-4f89-8ffa-16e053feb1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:29.011813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e71626-7624-4f89-8ffa-16e053feb1eb admin admin] PATH_INFO: `/v3/projects/93306ba6fcce457d9a839355ad89a897` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:29.015115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e71626-7624-4f89-8ffa-16e053feb1eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=93306ba6fcce457d9a839355ad89a897)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:29.017152 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 909/1851] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:28 2022] PUT /identity/v3/projects/40b5a74a98374cf999102f7cf16957d7/users/1102f5f98bf545ada39c019f45f2f413/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:29.018125 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e71626-7624-4f89-8ffa-16e053feb1eb admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:29.023540 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1826fa-93d1-4e98-8603-ee140570bf33 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:29.024122 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1826fa-93d1-4e98-8603-ee140570bf33 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:29.024625 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1826fa-93d1-4e98-8603-ee140570bf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:29.064352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-fe1826fa-93d1-4e98-8603-ee140570bf33 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:29.068592 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-fe1826fa-93d1-4e98-8603-ee140570bf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:29.181710 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 910/1852] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:29.226424 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1826fa-93d1-4e98-8603-ee140570bf33 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:29.226658 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-53e71626-7624-4f89-8ffa-16e053feb1eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93306ba6fcce457d9a839355ad89a897', 'request_id': 'req-53e71626-7624-4f89-8ffa-16e053feb1eb'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:29.226789 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-fe1826fa-93d1-4e98-8603-ee140570bf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:29.237322 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-53e71626-7624-4f89-8ffa-16e053feb1eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93306ba6fcce457d9a839355ad89a897', 'request_id': 'req-53e71626-7624-4f89-8ffa-16e053feb1eb'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:29.247604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-53e71626-7624-4f89-8ffa-16e053feb1eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93306ba6fcce457d9a839355ad89a897', 'request_id': 'req-53e71626-7624-4f89-8ffa-16e053feb1eb'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:29.312840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1826fa-93d1-4e98-8603-ee140570bf33 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:29.355587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-53e71626-7624-4f89-8ffa-16e053feb1eb admin admin] The token cache is being invalidate because project 93306ba6fcce457d9a839355ad89a897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:44:29.355928 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-53e71626-7624-4f89-8ffa-16e053feb1eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:29.356591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-53e71626-7624-4f89-8ffa-16e053feb1eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:29.361553 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 943/1853] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:44:28 2022] DELETE /identity/v3/projects/93306ba6fcce457d9a839355ad89a897 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:29.364394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472f602-c51f-4c9e-80b2-c4463081f9fe None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:29.364685 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472f602-c51f-4c9e-80b2-c4463081f9fe None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:29.365647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472f602-c51f-4c9e-80b2-c4463081f9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:29.406982 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-8472f602-c51f-4c9e-80b2-c4463081f9fe None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:29.408223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8472f602-c51f-4c9e-80b2-c4463081f9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:29.440104 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bc179-331b-42d0-b318-ed43bf10764e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:29.440104 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bc179-331b-42d0-b318-ed43bf10764e service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:29.440104 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bc179-331b-42d0-b318-ed43bf10764e service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:29.440104 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bc179-331b-42d0-b318-ed43bf10764e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:29.440418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-159bc179-331b-42d0-b318-ed43bf10764e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:29.519875 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 944/1854] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:29.532571 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527804e-2320-46f9-9e9e-0c22de2a211e None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:29.533535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527804e-2320-46f9-9e9e-0c22de2a211e None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:29.536484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527804e-2320-46f9-9e9e-0c22de2a211e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:29.547203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159bc179-331b-42d0-b318-ed43bf10764e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:29.548931 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159bc179-331b-42d0-b318-ed43bf10764e service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:29.570371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 911/1855] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:44:29.572561 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-2527804e-2320-46f9-9e9e-0c22de2a211e None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:29.574188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-2527804e-2320-46f9-9e9e-0c22de2a211e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:29.584921 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 945/1856] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:29.589185 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bc179-331b-42d0-b318-ed43bf10764e service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:29.599936 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bc179-331b-42d0-b318-ed43bf10764e service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:29.606910 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cdd4e4-6ea4-46ac-929a-67c2a4a0040d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:29.606910 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdd4e4-6ea4-46ac-929a-67c2a4a0040d admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:29.606910 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdd4e4-6ea4-46ac-929a-67c2a4a0040d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:29.606910 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdd4e4-6ea4-46ac-929a-67c2a4a0040d admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:29.607763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cdd4e4-6ea4-46ac-929a-67c2a4a0040d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:44:29.608203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cdd4e4-6ea4-46ac-929a-67c2a4a0040d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:29.610129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cdd4e4-6ea4-46ac-929a-67c2a4a0040d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:29.627080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 912/1857] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:44:29.628376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527804e-2320-46f9-9e9e-0c22de2a211e None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:29.642900 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527804e-2320-46f9-9e9e-0c22de2a211e None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:29.648914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d9df0-a224-44fb-a24e-b1be88ca93ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:29.649706 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d9df0-a224-44fb-a24e-b1be88ca93ee admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:29.650645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d9df0-a224-44fb-a24e-b1be88ca93ee admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:29.650645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d9df0-a224-44fb-a24e-b1be88ca93ee admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:29.651265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d9df0-a224-44fb-a24e-b1be88ca93ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:29.653432 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d9df0-a224-44fb-a24e-b1be88ca93ee admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:29.791919 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 946/1858] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:44:29 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:29.792821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cdd4e4-6ea4-46ac-929a-67c2a4a0040d admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:29.811450 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cdd4e4-6ea4-46ac-929a-67c2a4a0040d admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:29.889786 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb263a8-3915-479f-a0a0-6ff57f606059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:29.891049 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb263a8-3915-479f-a0a0-6ff57f606059 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:29.891649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb263a8-3915-479f-a0a0-6ff57f606059 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:29.891931 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb263a8-3915-479f-a0a0-6ff57f606059 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:29.892614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb263a8-3915-479f-a0a0-6ff57f606059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:29.894552 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb263a8-3915-479f-a0a0-6ff57f606059 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:30.023098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 913/1859] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:29 2022] POST /identity/v3/users => generated 292 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:30.023778 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d9df0-a224-44fb-a24e-b1be88ca93ee admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:30.034818 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d9df0-a224-44fb-a24e-b1be88ca93ee admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:30.040834 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695779e-5508-4fc7-9e47-01f5d4df686d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:30.042115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695779e-5508-4fc7-9e47-01f5d4df686d admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:30.042115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695779e-5508-4fc7-9e47-01f5d4df686d admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:30.042526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695779e-5508-4fc7-9e47-01f5d4df686d admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:30.043240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695779e-5508-4fc7-9e47-01f5d4df686d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:30.045299 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695779e-5508-4fc7-9e47-01f5d4df686d admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:30.060998 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 947/1860] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:30.063203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb263a8-3915-479f-a0a0-6ff57f606059 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:30.073904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb263a8-3915-479f-a0a0-6ff57f606059 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:30.078895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee21f9-589d-4c9e-8a7d-5183acf6ae0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:30.079502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee21f9-589d-4c9e-8a7d-5183acf6ae0f admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:30.079782 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee21f9-589d-4c9e-8a7d-5183acf6ae0f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:30.080053 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee21f9-589d-4c9e-8a7d-5183acf6ae0f admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:30.080726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee21f9-589d-4c9e-8a7d-5183acf6ae0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:30.082312 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee21f9-589d-4c9e-8a7d-5183acf6ae0f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:30.093910 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 914/1861] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:30.098325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695779e-5508-4fc7-9e47-01f5d4df686d admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:30.109549 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695779e-5508-4fc7-9e47-01f5d4df686d admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:30.114331 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4e186-9ec0-40f2-98db-ea8f0784a383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:30.117722 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4e186-9ec0-40f2-98db-ea8f0784a383 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:30.117722 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4e186-9ec0-40f2-98db-ea8f0784a383 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:30.117722 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4e186-9ec0-40f2-98db-ea8f0784a383 admin admin] PATH_INFO: `/v3/projects/24f3763760bd47c593175683f2a32eb4/users/eb0a771e710b4f6dbaf1f3e09d9bd2c6/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:30.133682 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4e186-9ec0-40f2-98db-ea8f0784a383 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24f3763760bd47c593175683f2a32eb4, user_id=eb0a771e710b4f6dbaf1f3e09d9bd2c6, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:30.136581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4e186-9ec0-40f2-98db-ea8f0784a383 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:30.205711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 948/1862] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:30 2022] PUT /identity/v3/projects/24f3763760bd47c593175683f2a32eb4/users/eb0a771e710b4f6dbaf1f3e09d9bd2c6/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:30.217639 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c45c9-b2aa-4c95-994d-007135f29ced None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:30.218241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c45c9-b2aa-4c95-994d-007135f29ced None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:30.218674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c45c9-b2aa-4c95-994d-007135f29ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:30.256636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-c61c45c9-b2aa-4c95-994d-007135f29ced None None] MFA Rules not processed for user `eb0a771e710b4f6dbaf1f3e09d9bd2c6`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:30.258198 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-c61c45c9-b2aa-4c95-994d-007135f29ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:30.337006 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 915/1863] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:30.351140 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4e186-9ec0-40f2-98db-ea8f0784a383 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:30.353004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b0f4e186-9ec0-40f2-98db-ea8f0784a383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:30.412944 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4e186-9ec0-40f2-98db-ea8f0784a383 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:30.568597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba53a2-6070-42be-adba-fc8c74184185 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:30.568597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba53a2-6070-42be-adba-fc8c74184185 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:30.568597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba53a2-6070-42be-adba-fc8c74184185 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:30.568597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba53a2-6070-42be-adba-fc8c74184185 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:30.569692 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba53a2-6070-42be-adba-fc8c74184185 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:30.569692 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba53a2-6070-42be-adba-fc8c74184185 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:30.569692 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 949/1864] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:31.033738 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:31.108399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:31.783465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957a677-e868-464e-ba0a-eba94cd7d587 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:31.783779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957a677-e868-464e-ba0a-eba94cd7d587 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:31.784055 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957a677-e868-464e-ba0a-eba94cd7d587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:31.819191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-b957a677-e868-464e-ba0a-eba94cd7d587 None None] MFA Rules not processed for user `1102f5f98bf545ada39c019f45f2f413`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:31.820514 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b957a677-e868-464e-ba0a-eba94cd7d587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:31.890626 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 916/1865] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:31.894792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba53a2-6070-42be-adba-fc8c74184185 service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:31.896359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f8ba53a2-6070-42be-adba-fc8c74184185 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:31.964197 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba53a2-6070-42be-adba-fc8c74184185 service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:31.969236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf1a3d-fb79-451d-998d-5c6b1a29de5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:31.969850 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf1a3d-fb79-451d-998d-5c6b1a29de5b admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:31.970120 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf1a3d-fb79-451d-998d-5c6b1a29de5b admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:31.970386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf1a3d-fb79-451d-998d-5c6b1a29de5b admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:31.971021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf1a3d-fb79-451d-998d-5c6b1a29de5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:31.972924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf1a3d-fb79-451d-998d-5c6b1a29de5b admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:32.059138 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 950/1866] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:44:31 2022] POST /identity/v3/projects => generated 371 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:32.062749 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957a677-e868-464e-ba0a-eba94cd7d587 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:32.074260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957a677-e868-464e-ba0a-eba94cd7d587 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:32.147235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91d711-f137-48f3-8200-8d44f8f6a499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:32.148206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91d711-f137-48f3-8200-8d44f8f6a499 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:32.148526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91d711-f137-48f3-8200-8d44f8f6a499 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:32.148858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91d711-f137-48f3-8200-8d44f8f6a499 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:32.149663 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac91d711-f137-48f3-8200-8d44f8f6a499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:32.151853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac91d711-f137-48f3-8200-8d44f8f6a499 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:32.273792 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 917/1867] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:32 2022] POST /identity/v3/users => generated 292 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:44:32.277936 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf1a3d-fb79-451d-998d-5c6b1a29de5b admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:32.289438 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf1a3d-fb79-451d-998d-5c6b1a29de5b admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:32.294360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98fe55-dcf3-4d38-b7b2-545cb5b9ae1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:32.294974 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98fe55-dcf3-4d38-b7b2-545cb5b9ae1c admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:32.295252 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98fe55-dcf3-4d38-b7b2-545cb5b9ae1c admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:32.295527 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98fe55-dcf3-4d38-b7b2-545cb5b9ae1c admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:32.296191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb98fe55-dcf3-4d38-b7b2-545cb5b9ae1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:32.297804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb98fe55-dcf3-4d38-b7b2-545cb5b9ae1c admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:32.310057 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 951/1868] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:44:32.314525 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91d711-f137-48f3-8200-8d44f8f6a499 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:32.326166 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91d711-f137-48f3-8200-8d44f8f6a499 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:32.332014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f83bce-2352-443b-9cd6-4f71649aa6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:32.332192 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f83bce-2352-443b-9cd6-4f71649aa6ca admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:32.332448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f83bce-2352-443b-9cd6-4f71649aa6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:32.332721 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f83bce-2352-443b-9cd6-4f71649aa6ca admin admin] PATH_INFO: `/v3/projects/0d19b10cb4b74e2d9300cc0db76035e0/users/fb6d101335a046498de9cda35ab3bc03/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:32.350597 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f83bce-2352-443b-9cd6-4f71649aa6ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d19b10cb4b74e2d9300cc0db76035e0, user_id=fb6d101335a046498de9cda35ab3bc03, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:32.359254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f83bce-2352-443b-9cd6-4f71649aa6ca admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:32.387977 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:32.496799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 918/1869] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:32 2022] PUT /identity/v3/projects/0d19b10cb4b74e2d9300cc0db76035e0/users/fb6d101335a046498de9cda35ab3bc03/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:32.510729 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98fe55-dcf3-4d38-b7b2-545cb5b9ae1c admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:32.520534 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98fe55-dcf3-4d38-b7b2-545cb5b9ae1c admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:32.580740 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7912681-5acf-4730-ba02-2b3596b32088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:32.587966 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7912681-5acf-4730-ba02-2b3596b32088 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:32.587966 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7912681-5acf-4730-ba02-2b3596b32088 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:32.587966 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7912681-5acf-4730-ba02-2b3596b32088 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:32.587966 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7912681-5acf-4730-ba02-2b3596b32088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:32.588453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7912681-5acf-4730-ba02-2b3596b32088 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:32.607784 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 952/1870] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:32.612186 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f83bce-2352-443b-9cd6-4f71649aa6ca admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:32.623093 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f83bce-2352-443b-9cd6-4f71649aa6ca admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:32.628478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97286b3b-999c-4d4f-83ee-84a495a4cfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:32.634419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97286b3b-999c-4d4f-83ee-84a495a4cfce admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:32.634419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97286b3b-999c-4d4f-83ee-84a495a4cfce admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:32.634419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97286b3b-999c-4d4f-83ee-84a495a4cfce admin admin] PATH_INFO: `/v3/projects/0d19b10cb4b74e2d9300cc0db76035e0/users/fb6d101335a046498de9cda35ab3bc03/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:32.634419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97286b3b-999c-4d4f-83ee-84a495a4cfce admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d19b10cb4b74e2d9300cc0db76035e0, user_id=fb6d101335a046498de9cda35ab3bc03, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:32.634949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97286b3b-999c-4d4f-83ee-84a495a4cfce admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:32.652475 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 919/1871] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:32 2022] PUT /identity/v3/projects/0d19b10cb4b74e2d9300cc0db76035e0/users/fb6d101335a046498de9cda35ab3bc03/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:44:33.280676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:33.994239 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:34.133629 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b766585-94ac-4535-96cb-2ac8eba95fdd None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:34.133915 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b766585-94ac-4535-96cb-2ac8eba95fdd None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:34.134193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b766585-94ac-4535-96cb-2ac8eba95fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:34.166636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-3b766585-94ac-4535-96cb-2ac8eba95fdd None None] MFA Rules not processed for user `fb6d101335a046498de9cda35ab3bc03`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:34.168415 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3b766585-94ac-4535-96cb-2ac8eba95fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:34.251027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 953/1872] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:34.263419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97286b3b-999c-4d4f-83ee-84a495a4cfce admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:34.271621 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-97286b3b-999c-4d4f-83ee-84a495a4cfce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:34.325427 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97286b3b-999c-4d4f-83ee-84a495a4cfce admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:34.329504 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91e2a3-7eb4-4f73-9075-6d3e593adf94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:34.330112 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91e2a3-7eb4-4f73-9075-6d3e593adf94 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:34.330394 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91e2a3-7eb4-4f73-9075-6d3e593adf94 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:34.330667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91e2a3-7eb4-4f73-9075-6d3e593adf94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:34.394348 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91e2a3-7eb4-4f73-9075-6d3e593adf94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:34.395886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91e2a3-7eb4-4f73-9075-6d3e593adf94 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:34.405876 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 920/1873] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:34.537377 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:35.090531 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:35.542119 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:35.782050 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:38.305739 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:38.348089 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b766585-94ac-4535-96cb-2ac8eba95fdd None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:38.358680 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b766585-94ac-4535-96cb-2ac8eba95fdd None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:38.362935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91e2a3-7eb4-4f73-9075-6d3e593adf94 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:38.363405 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee58a55-2e63-4f9a-bc3d-81c67f0b15f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:38.368959 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee58a55-2e63-4f9a-bc3d-81c67f0b15f8 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:38.368959 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee58a55-2e63-4f9a-bc3d-81c67f0b15f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:38.368959 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee58a55-2e63-4f9a-bc3d-81c67f0b15f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:38.374513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee58a55-2e63-4f9a-bc3d-81c67f0b15f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:38.375945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee58a55-2e63-4f9a-bc3d-81c67f0b15f8 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:38.399067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 954/1874] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:44:38.516562 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91e2a3-7eb4-4f73-9075-6d3e593adf94 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:38.531963 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba41626-fdb9-460d-a8dd-22f14847ce43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:38.532897 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba41626-fdb9-460d-a8dd-22f14847ce43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:38.533180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba41626-fdb9-460d-a8dd-22f14847ce43 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:38.533902 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba41626-fdb9-460d-a8dd-22f14847ce43 admin admin] PATH_INFO: `/v3/users/eb0a771e710b4f6dbaf1f3e09d9bd2c6` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:38.537022 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba41626-fdb9-460d-a8dd-22f14847ce43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb0a771e710b4f6dbaf1f3e09d9bd2c6)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:38.542566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba41626-fdb9-460d-a8dd-22f14847ce43 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:38.655352 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139a78d-244c-4228-8c63-b72ebb62b297 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:38.655636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139a78d-244c-4228-8c63-b72ebb62b297 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:38.655918 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139a78d-244c-4228-8c63-b72ebb62b297 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:38.657650 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 955/1875] 10.222.0.59 () {54 vars in 921 bytes} [Mon Aug 29 22:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:44:38.782469 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ab7f2-4215-4216-a28b-8214d50d0713 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:38.782744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ab7f2-4215-4216-a28b-8214d50d0713 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:38.783017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ab7f2-4215-4216-a28b-8214d50d0713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:38.805534 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-943ab7f2-4215-4216-a28b-8214d50d0713 None None] MFA Rules not processed for user `7f21105679f245a19ea4850e6c010dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:38.806729 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-943ab7f2-4215-4216-a28b-8214d50d0713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:38.824048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-cba41626-fdb9-460d-a8dd-22f14847ce43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb0a771e710b4f6dbaf1f3e09d9bd2c6'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:38.832149 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-cba41626-fdb9-460d-a8dd-22f14847ce43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb0a771e710b4f6dbaf1f3e09d9bd2c6'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:38.834562 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-cba41626-fdb9-460d-a8dd-22f14847ce43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb0a771e710b4f6dbaf1f3e09d9bd2c6'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:38.851227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-cba41626-fdb9-460d-a8dd-22f14847ce43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb0a771e710b4f6dbaf1f3e09d9bd2c6'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:38.852098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-cba41626-fdb9-460d-a8dd-22f14847ce43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb0a771e710b4f6dbaf1f3e09d9bd2c6'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:38.890100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 956/1876] 10.222.0.59 () {58 vars in 1018 bytes} [Mon Aug 29 22:44:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:38.903919 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ab7f2-4215-4216-a28b-8214d50d0713 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:38.905371 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-943ab7f2-4215-4216-a28b-8214d50d0713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:38.933504 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ab7f2-4215-4216-a28b-8214d50d0713 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:38.941856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420382d-e7c9-478c-8e96-e92b412aa2c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:38.943202 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420382d-e7c9-478c-8e96-e92b412aa2c5 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:38.943604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420382d-e7c9-478c-8e96-e92b412aa2c5 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:38.943884 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420382d-e7c9-478c-8e96-e92b412aa2c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:38.953302 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420382d-e7c9-478c-8e96-e92b412aa2c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:38.955037 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420382d-e7c9-478c-8e96-e92b412aa2c5 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:38.968452 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 957/1877] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:44:38.985791 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 921/1878] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:44:38 2022] DELETE /identity/v3/users/eb0a771e710b4f6dbaf1f3e09d9bd2c6 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:38.990851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906fc34-1db4-4354-bb00-bb4c1ac0144b None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:38.991136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906fc34-1db4-4354-bb00-bb4c1ac0144b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:38.991404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906fc34-1db4-4354-bb00-bb4c1ac0144b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:39.017239 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-8906fc34-1db4-4354-bb00-bb4c1ac0144b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:39.017239 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8906fc34-1db4-4354-bb00-bb4c1ac0144b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:39.072199 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 958/1879] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:39.082051 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba41626-fdb9-460d-a8dd-22f14847ce43 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:39.083889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-cba41626-fdb9-460d-a8dd-22f14847ce43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:39.151962 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba41626-fdb9-460d-a8dd-22f14847ce43 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:39.156017 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9840e-4e51-44ce-a73e-69c47f52673d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:39.156713 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9840e-4e51-44ce-a73e-69c47f52673d service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:39.157053 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9840e-4e51-44ce-a73e-69c47f52673d service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:39.157463 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9840e-4e51-44ce-a73e-69c47f52673d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:39.168013 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9840e-4e51-44ce-a73e-69c47f52673d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:39.169711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9840e-4e51-44ce-a73e-69c47f52673d service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:39.180561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 922/1880] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:39.262103 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:39.819487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906fc34-1db4-4354-bb00-bb4c1ac0144b None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:39.821225 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8906fc34-1db4-4354-bb00-bb4c1ac0144b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:39.834621 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906fc34-1db4-4354-bb00-bb4c1ac0144b None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:39.844331 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4119d-6e62-4adf-a30b-31f71b4a97f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:39.845294 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4119d-6e62-4adf-a30b-31f71b4a97f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:39.847185 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4119d-6e62-4adf-a30b-31f71b4a97f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:39.847185 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4119d-6e62-4adf-a30b-31f71b4a97f1 admin admin] PATH_INFO: `/v3/projects/24f3763760bd47c593175683f2a32eb4` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:39.848047 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4119d-6e62-4adf-a30b-31f71b4a97f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24f3763760bd47c593175683f2a32eb4)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:39.851359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4119d-6e62-4adf-a30b-31f71b4a97f1 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:40.014412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-cae4119d-6e62-4adf-a30b-31f71b4a97f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24f3763760bd47c593175683f2a32eb4', 'request_id': 'req-cae4119d-6e62-4adf-a30b-31f71b4a97f1'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:40.024523 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-cae4119d-6e62-4adf-a30b-31f71b4a97f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24f3763760bd47c593175683f2a32eb4', 'request_id': 'req-cae4119d-6e62-4adf-a30b-31f71b4a97f1'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:40.035678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-cae4119d-6e62-4adf-a30b-31f71b4a97f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24f3763760bd47c593175683f2a32eb4', 'request_id': 'req-cae4119d-6e62-4adf-a30b-31f71b4a97f1'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:40.135687 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-cae4119d-6e62-4adf-a30b-31f71b4a97f1 admin admin] The token cache is being invalidate because project 24f3763760bd47c593175683f2a32eb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:44:40.136295 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-cae4119d-6e62-4adf-a30b-31f71b4a97f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:40.136989 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-cae4119d-6e62-4adf-a30b-31f71b4a97f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:40.138585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 959/1881] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:44:39 2022] DELETE /identity/v3/projects/24f3763760bd47c593175683f2a32eb4 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:40.144196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f216c67-54bf-4a0c-aab9-b7bc0a9f39f3 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:40.144702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f216c67-54bf-4a0c-aab9-b7bc0a9f39f3 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:40.146270 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f216c67-54bf-4a0c-aab9-b7bc0a9f39f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:40.169357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-9f216c67-54bf-4a0c-aab9-b7bc0a9f39f3 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:40.170572 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-9f216c67-54bf-4a0c-aab9-b7bc0a9f39f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:40.244466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 923/1882] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:40.250633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6706f4-b6cb-43f7-976d-c76790c386a4 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:40.250916 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6706f4-b6cb-43f7-976d-c76790c386a4 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:40.251283 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6706f4-b6cb-43f7-976d-c76790c386a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:40.274928 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:40.296627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-9e6706f4-b6cb-43f7-976d-c76790c386a4 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:40.302311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-9e6706f4-b6cb-43f7-976d-c76790c386a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:40.314909 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 960/1883] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:40.316922 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f216c67-54bf-4a0c-aab9-b7bc0a9f39f3 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:40.329509 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f216c67-54bf-4a0c-aab9-b7bc0a9f39f3 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:40.335586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b721822-b002-4fe2-b2b7-961aad574c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:40.336568 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b721822-b002-4fe2-b2b7-961aad574c07 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:40.336850 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b721822-b002-4fe2-b2b7-961aad574c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:40.337126 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b721822-b002-4fe2-b2b7-961aad574c07 admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:40.337941 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b721822-b002-4fe2-b2b7-961aad574c07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:44:40.338250 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b721822-b002-4fe2-b2b7-961aad574c07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:40.339863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b721822-b002-4fe2-b2b7-961aad574c07 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:40.352870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 924/1884] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:44:40.357588 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6706f4-b6cb-43f7-976d-c76790c386a4 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:40.368039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6706f4-b6cb-43f7-976d-c76790c386a4 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:40.376061 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4968d-ee71-483a-a8e1-d797ee10988d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:40.376648 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4968d-ee71-483a-a8e1-d797ee10988d admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:40.376911 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4968d-ee71-483a-a8e1-d797ee10988d admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:40.377339 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4968d-ee71-483a-a8e1-d797ee10988d admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:40.378168 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a4968d-ee71-483a-a8e1-d797ee10988d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:40.380781 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a4968d-ee71-483a-a8e1-d797ee10988d admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:40.539251 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 961/1885] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:44:40 2022] POST /identity/v3/projects => generated 363 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:44:40.542271 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b721822-b002-4fe2-b2b7-961aad574c07 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:40.564297 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b721822-b002-4fe2-b2b7-961aad574c07 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:40.664639 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0858a42-6434-41b0-b463-6bcbe7467759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:40.665884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0858a42-6434-41b0-b463-6bcbe7467759 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:40.666374 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0858a42-6434-41b0-b463-6bcbe7467759 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:40.666908 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0858a42-6434-41b0-b463-6bcbe7467759 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:40.667763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0858a42-6434-41b0-b463-6bcbe7467759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:40.671127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0858a42-6434-41b0-b463-6bcbe7467759 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:40.781918 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 925/1886] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:40 2022] POST /identity/v3/users => generated 282 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:40.787333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4968d-ee71-483a-a8e1-d797ee10988d admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:40.798041 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4968d-ee71-483a-a8e1-d797ee10988d admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:40.803244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d277c-1cb3-465d-8814-cf7260fb89e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:40.803885 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d277c-1cb3-465d-8814-cf7260fb89e4 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:40.804197 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d277c-1cb3-465d-8814-cf7260fb89e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:40.804474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d277c-1cb3-465d-8814-cf7260fb89e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:40.805287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d277c-1cb3-465d-8814-cf7260fb89e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:40.806901 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d277c-1cb3-465d-8814-cf7260fb89e4 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:40.818420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 962/1887] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:44:40.823078 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0858a42-6434-41b0-b463-6bcbe7467759 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:40.837377 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0858a42-6434-41b0-b463-6bcbe7467759 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:40.842627 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfb48a-14ac-4c09-b71d-1da5272025dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:40.843434 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfb48a-14ac-4c09-b71d-1da5272025dd admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:40.843716 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfb48a-14ac-4c09-b71d-1da5272025dd admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:40.844743 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfb48a-14ac-4c09-b71d-1da5272025dd admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:40.844938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfb48a-14ac-4c09-b71d-1da5272025dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:40.846794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfb48a-14ac-4c09-b71d-1da5272025dd admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:40.878297 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 926/1888] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:40.881242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d277c-1cb3-465d-8814-cf7260fb89e4 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:40.893084 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d277c-1cb3-465d-8814-cf7260fb89e4 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:40.899214 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ff424-db6f-4f08-aed1-45a93f486732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:40.900734 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ff424-db6f-4f08-aed1-45a93f486732 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:40.901289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ff424-db6f-4f08-aed1-45a93f486732 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:40.901762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ff424-db6f-4f08-aed1-45a93f486732 admin admin] PATH_INFO: `/v3/projects/f1ca3c31fa114ea5a6628c510e5de3fd/users/ab35de2c5ea84103b3e4e08aeeb3f857/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:40.939791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443ff424-db6f-4f08-aed1-45a93f486732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1ca3c31fa114ea5a6628c510e5de3fd, user_id=ab35de2c5ea84103b3e4e08aeeb3f857, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:40.943370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443ff424-db6f-4f08-aed1-45a93f486732 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:41.128621 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 963/1889] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:40 2022] PUT /identity/v3/projects/f1ca3c31fa114ea5a6628c510e5de3fd/users/ab35de2c5ea84103b3e4e08aeeb3f857/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:44:41.139502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224aa6f-8a50-414b-af61-3fe0c881a0d4 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:41.139783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224aa6f-8a50-414b-af61-3fe0c881a0d4 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:41.140052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224aa6f-8a50-414b-af61-3fe0c881a0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:41.221412 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-2224aa6f-8a50-414b-af61-3fe0c881a0d4 None None] MFA Rules not processed for user `ab35de2c5ea84103b3e4e08aeeb3f857`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:41.222649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-2224aa6f-8a50-414b-af61-3fe0c881a0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:41.276818 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:41.327221 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 927/1890] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:41.346064 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ff424-db6f-4f08-aed1-45a93f486732 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:41.350467 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-443ff424-db6f-4f08-aed1-45a93f486732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:41.446113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ff424-db6f-4f08-aed1-45a93f486732 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:41.452486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d93f6e-197a-493e-b4fc-a419ab338820 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:41.459206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d93f6e-197a-493e-b4fc-a419ab338820 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:41.459206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d93f6e-197a-493e-b4fc-a419ab338820 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:41.459206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d93f6e-197a-493e-b4fc-a419ab338820 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:41.464533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d93f6e-197a-493e-b4fc-a419ab338820 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:41.466066 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d93f6e-197a-493e-b4fc-a419ab338820 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:41.477442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 964/1891] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:44:42.346067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:44.103981 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:44.381937 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:45.233438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:45.279280 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:46.148550 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:46.181956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:46.353935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:46.713926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224aa6f-8a50-414b-af61-3fe0c881a0d4 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:46.715323 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-2224aa6f-8a50-414b-af61-3fe0c881a0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:46.731615 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:46.790334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224aa6f-8a50-414b-af61-3fe0c881a0d4 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:46.795361 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e2b5f-679a-4e58-a499-0123b5b7bfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:46.797319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e2b5f-679a-4e58-a499-0123b5b7bfd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:46.797319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e2b5f-679a-4e58-a499-0123b5b7bfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:46.797319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e2b5f-679a-4e58-a499-0123b5b7bfd9 admin admin] PATH_INFO: `/v3/users/92175ee3c02d4c919bfdbb26d5e4a4de` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:46.798708 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e2b5f-679a-4e58-a499-0123b5b7bfd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92175ee3c02d4c919bfdbb26d5e4a4de)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:46.802133 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e2b5f-679a-4e58-a499-0123b5b7bfd9 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:47.041377 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:47.077734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-8d6e2b5f-679a-4e58-a499-0123b5b7bfd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92175ee3c02d4c919bfdbb26d5e4a4de'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:47.088694 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-8d6e2b5f-679a-4e58-a499-0123b5b7bfd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92175ee3c02d4c919bfdbb26d5e4a4de'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:47.089554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-8d6e2b5f-679a-4e58-a499-0123b5b7bfd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92175ee3c02d4c919bfdbb26d5e4a4de'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:47.104488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-8d6e2b5f-679a-4e58-a499-0123b5b7bfd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92175ee3c02d4c919bfdbb26d5e4a4de'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:47.105284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-8d6e2b5f-679a-4e58-a499-0123b5b7bfd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92175ee3c02d4c919bfdbb26d5e4a4de'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:47.190120 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 928/1892] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:44:46 2022] DELETE /identity/v3/users/92175ee3c02d4c919bfdbb26d5e4a4de => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:47.190229 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85975088-a563-4e9e-9f0a-ce4f4e7fc54c None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:47.190229 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85975088-a563-4e9e-9f0a-ce4f4e7fc54c None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:47.190229 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85975088-a563-4e9e-9f0a-ce4f4e7fc54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:47.208618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-85975088-a563-4e9e-9f0a-ce4f4e7fc54c None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:47.210284 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-85975088-a563-4e9e-9f0a-ce4f4e7fc54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:47.296642 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 965/1893] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:47.312893 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e2b5f-679a-4e58-a499-0123b5b7bfd9 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:47.314407 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-8d6e2b5f-679a-4e58-a499-0123b5b7bfd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:47.396819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e2b5f-679a-4e58-a499-0123b5b7bfd9 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:47.401195 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21c8c0-60d1-464a-99eb-52f7b0f37dde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:47.401978 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21c8c0-60d1-464a-99eb-52f7b0f37dde service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:47.402590 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21c8c0-60d1-464a-99eb-52f7b0f37dde service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:47.403826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21c8c0-60d1-464a-99eb-52f7b0f37dde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:47.436508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21c8c0-60d1-464a-99eb-52f7b0f37dde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:47.438250 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21c8c0-60d1-464a-99eb-52f7b0f37dde service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:47.445395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:47.454020 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 929/1894] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:47.685102 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:48.370192 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85975088-a563-4e9e-9f0a-ce4f4e7fc54c None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:48.372680 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-85975088-a563-4e9e-9f0a-ce4f4e7fc54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:48.421248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85975088-a563-4e9e-9f0a-ce4f4e7fc54c None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:48.430987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318358eb-d170-4525-928c-136b2caa7c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:48.432674 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318358eb-d170-4525-928c-136b2caa7c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:48.433233 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318358eb-d170-4525-928c-136b2caa7c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:48.433746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318358eb-d170-4525-928c-136b2caa7c35 admin admin] PATH_INFO: `/v3/projects/730a8bf0156545a2b8a0d926002bf809` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:48.435166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318358eb-d170-4525-928c-136b2caa7c35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=730a8bf0156545a2b8a0d926002bf809)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:48.438173 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318358eb-d170-4525-928c-136b2caa7c35 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:48.605972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-318358eb-d170-4525-928c-136b2caa7c35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '730a8bf0156545a2b8a0d926002bf809', 'request_id': 'req-318358eb-d170-4525-928c-136b2caa7c35'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:48.612434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-318358eb-d170-4525-928c-136b2caa7c35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '730a8bf0156545a2b8a0d926002bf809', 'request_id': 'req-318358eb-d170-4525-928c-136b2caa7c35'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:48.628646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-318358eb-d170-4525-928c-136b2caa7c35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '730a8bf0156545a2b8a0d926002bf809', 'request_id': 'req-318358eb-d170-4525-928c-136b2caa7c35'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:48.725603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-318358eb-d170-4525-928c-136b2caa7c35 admin admin] The token cache is being invalidate because project 730a8bf0156545a2b8a0d926002bf809 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:44:48.725898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-318358eb-d170-4525-928c-136b2caa7c35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:48.726539 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-318358eb-d170-4525-928c-136b2caa7c35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:48.728290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 966/1895] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:44:48 2022] DELETE /identity/v3/projects/730a8bf0156545a2b8a0d926002bf809 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:48.732454 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21c8c0-60d1-464a-99eb-52f7b0f37dde service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:48.752330 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21c8c0-60d1-464a-99eb-52f7b0f37dde service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:48.835102 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005b0e8-214e-4378-90c2-091578aee6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:48.835740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005b0e8-214e-4378-90c2-091578aee6d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:48.836015 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005b0e8-214e-4378-90c2-091578aee6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:48.836340 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005b0e8-214e-4378-90c2-091578aee6d9 admin admin] PATH_INFO: `/v3/users/2cba78a0b00446eb99cf55ab1f42ebba` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:48.837714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2005b0e8-214e-4378-90c2-091578aee6d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cba78a0b00446eb99cf55ab1f42ebba)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:48.839997 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2005b0e8-214e-4378-90c2-091578aee6d9 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:49.101256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2005b0e8-214e-4378-90c2-091578aee6d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cba78a0b00446eb99cf55ab1f42ebba'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:49.111460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2005b0e8-214e-4378-90c2-091578aee6d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cba78a0b00446eb99cf55ab1f42ebba'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:49.118406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2005b0e8-214e-4378-90c2-091578aee6d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cba78a0b00446eb99cf55ab1f42ebba'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:49.129910 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2005b0e8-214e-4378-90c2-091578aee6d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cba78a0b00446eb99cf55ab1f42ebba'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:49.131167 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2005b0e8-214e-4378-90c2-091578aee6d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cba78a0b00446eb99cf55ab1f42ebba'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:49.257715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 930/1896] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:44:48 2022] DELETE /identity/v3/users/2cba78a0b00446eb99cf55ab1f42ebba => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:49.382109 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:49.743866 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318358eb-d170-4525-928c-136b2caa7c35 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:49.745398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-318358eb-d170-4525-928c-136b2caa7c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:49.760921 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318358eb-d170-4525-928c-136b2caa7c35 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:49.825499 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97027e81-f733-444e-97ee-e448e04526d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:49.826355 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97027e81-f733-444e-97ee-e448e04526d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:49.826641 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97027e81-f733-444e-97ee-e448e04526d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:49.826914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97027e81-f733-444e-97ee-e448e04526d4 admin admin] PATH_INFO: `/v3/projects/6518fcb59dd54cf7b3fc9a6558377b32` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:49.828056 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97027e81-f733-444e-97ee-e448e04526d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6518fcb59dd54cf7b3fc9a6558377b32)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:49.830024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97027e81-f733-444e-97ee-e448e04526d4 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:50.011036 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-97027e81-f733-444e-97ee-e448e04526d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6518fcb59dd54cf7b3fc9a6558377b32', 'request_id': 'req-97027e81-f733-444e-97ee-e448e04526d4'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:50.028262 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-97027e81-f733-444e-97ee-e448e04526d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6518fcb59dd54cf7b3fc9a6558377b32', 'request_id': 'req-97027e81-f733-444e-97ee-e448e04526d4'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:50.028262 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-97027e81-f733-444e-97ee-e448e04526d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6518fcb59dd54cf7b3fc9a6558377b32', 'request_id': 'req-97027e81-f733-444e-97ee-e448e04526d4'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:50.141585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-97027e81-f733-444e-97ee-e448e04526d4 admin admin] The token cache is being invalidate because project 6518fcb59dd54cf7b3fc9a6558377b32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:44:50.141971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-97027e81-f733-444e-97ee-e448e04526d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:50.142660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-97027e81-f733-444e-97ee-e448e04526d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:50.144205 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 967/1897] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:44:49 2022] DELETE /identity/v3/projects/6518fcb59dd54cf7b3fc9a6558377b32 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:50.150166 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9091c5-a75f-4018-9a28-3943dc81e865 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:50.150666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9091c5-a75f-4018-9a28-3943dc81e865 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:50.150744 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9091c5-a75f-4018-9a28-3943dc81e865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:50.195347 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-3c9091c5-a75f-4018-9a28-3943dc81e865 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:50.195347 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3c9091c5-a75f-4018-9a28-3943dc81e865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:50.265521 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 931/1898] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:50.273526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac95c2a-d78b-4f95-9dc0-e92c6e330fed None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:50.273991 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac95c2a-d78b-4f95-9dc0-e92c6e330fed None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:50.274340 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac95c2a-d78b-4f95-9dc0-e92c6e330fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:50.303376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-6ac95c2a-d78b-4f95-9dc0-e92c6e330fed None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:50.304782 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6ac95c2a-d78b-4f95-9dc0-e92c6e330fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:50.321814 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 968/1899] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:50.326036 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9091c5-a75f-4018-9a28-3943dc81e865 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:50.337998 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9091c5-a75f-4018-9a28-3943dc81e865 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:50.343201 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0ad64-0fd6-49f1-b8ce-6aec2da2897c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:50.343830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0ad64-0fd6-49f1-b8ce-6aec2da2897c admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:50.344106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0ad64-0fd6-49f1-b8ce-6aec2da2897c admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:50.344410 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0ad64-0fd6-49f1-b8ce-6aec2da2897c admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:50.345097 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0ad64-0fd6-49f1-b8ce-6aec2da2897c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:44:50.345567 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0ad64-0fd6-49f1-b8ce-6aec2da2897c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:50.347342 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0ad64-0fd6-49f1-b8ce-6aec2da2897c admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:50.371142 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 932/1900] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:44:50.372846 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac95c2a-d78b-4f95-9dc0-e92c6e330fed None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:50.378225 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:50.387506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac95c2a-d78b-4f95-9dc0-e92c6e330fed None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:50.393439 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8895e-19b2-4773-9ee7-583d3de1df46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:50.394182 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8895e-19b2-4773-9ee7-583d3de1df46 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:50.396004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8895e-19b2-4773-9ee7-583d3de1df46 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:50.397401 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8895e-19b2-4773-9ee7-583d3de1df46 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:50.405968 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d8895e-19b2-4773-9ee7-583d3de1df46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:50.405968 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d8895e-19b2-4773-9ee7-583d3de1df46 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:50.526232 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 969/1901] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:44:50 2022] POST /identity/v3/projects => generated 397 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:50.530360 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0ad64-0fd6-49f1-b8ce-6aec2da2897c admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:50.542182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0ad64-0fd6-49f1-b8ce-6aec2da2897c admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:50.607046 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409edb6a-6c04-47c5-a0f5-d9e1102856af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:50.607678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409edb6a-6c04-47c5-a0f5-d9e1102856af admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:50.607975 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409edb6a-6c04-47c5-a0f5-d9e1102856af admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:50.608282 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409edb6a-6c04-47c5-a0f5-d9e1102856af admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:50.608907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409edb6a-6c04-47c5-a0f5-d9e1102856af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:50.610894 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409edb6a-6c04-47c5-a0f5-d9e1102856af admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:50.782457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 933/1902] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:50 2022] POST /identity/v3/users => generated 299 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:50.787129 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8895e-19b2-4773-9ee7-583d3de1df46 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:50.798068 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8895e-19b2-4773-9ee7-583d3de1df46 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:50.807028 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059700c-a230-4bf1-8082-5fbfca47f6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:50.807714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059700c-a230-4bf1-8082-5fbfca47f6c4 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:50.808104 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059700c-a230-4bf1-8082-5fbfca47f6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:50.808462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059700c-a230-4bf1-8082-5fbfca47f6c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:50.809181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059700c-a230-4bf1-8082-5fbfca47f6c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:50.810885 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059700c-a230-4bf1-8082-5fbfca47f6c4 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:50.827717 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 970/1903] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:50.828935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409edb6a-6c04-47c5-a0f5-d9e1102856af admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:50.840125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409edb6a-6c04-47c5-a0f5-d9e1102856af admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:50.846471 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dde0f6-99a4-4ab2-8620-4160fe088f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:50.847178 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dde0f6-99a4-4ab2-8620-4160fe088f0b admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:50.847519 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dde0f6-99a4-4ab2-8620-4160fe088f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:50.848417 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dde0f6-99a4-4ab2-8620-4160fe088f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:50.849355 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dde0f6-99a4-4ab2-8620-4160fe088f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:50.851038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dde0f6-99a4-4ab2-8620-4160fe088f0b admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:50.866541 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 934/1904] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:50.867678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059700c-a230-4bf1-8082-5fbfca47f6c4 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:50.877818 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059700c-a230-4bf1-8082-5fbfca47f6c4 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:50.882678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e3c6e-db9c-4b03-88e4-74fcb96cd3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:50.883354 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e3c6e-db9c-4b03-88e4-74fcb96cd3b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:50.883625 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e3c6e-db9c-4b03-88e4-74fcb96cd3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:50.884063 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e3c6e-db9c-4b03-88e4-74fcb96cd3b0 admin admin] PATH_INFO: `/v3/projects/7900f30129fa4a5bb015becb1149b4e4/users/1fa4f486d0c14d5cba39035a5c7985b6/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:50.899830 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e3c6e-db9c-4b03-88e4-74fcb96cd3b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7900f30129fa4a5bb015becb1149b4e4, user_id=1fa4f486d0c14d5cba39035a5c7985b6, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:50.902497 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e3c6e-db9c-4b03-88e4-74fcb96cd3b0 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:50.964839 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:51.032804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 971/1905] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:50 2022] PUT /identity/v3/projects/7900f30129fa4a5bb015becb1149b4e4/users/1fa4f486d0c14d5cba39035a5c7985b6/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:51.044058 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b6e21-c69b-4044-b790-21a9d3cec6d2 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:51.044175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b6e21-c69b-4044-b790-21a9d3cec6d2 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:51.044502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b6e21-c69b-4044-b790-21a9d3cec6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:51.077491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-6a3b6e21-c69b-4044-b790-21a9d3cec6d2 None None] MFA Rules not processed for user `1fa4f486d0c14d5cba39035a5c7985b6`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:51.078740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6a3b6e21-c69b-4044-b790-21a9d3cec6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:51.154533 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 935/1906] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:51 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:51.159231 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e3c6e-db9c-4b03-88e4-74fcb96cd3b0 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:51.240198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e3c6e-db9c-4b03-88e4-74fcb96cd3b0 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:51.250111 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8227d1-d0e7-4ed2-a419-10c5c0a9e4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:51.250687 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8227d1-d0e7-4ed2-a419-10c5c0a9e4b8 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:51.251435 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8227d1-d0e7-4ed2-a419-10c5c0a9e4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:51.251435 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8227d1-d0e7-4ed2-a419-10c5c0a9e4b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:51.251965 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8227d1-d0e7-4ed2-a419-10c5c0a9e4b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:51.254862 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8227d1-d0e7-4ed2-a419-10c5c0a9e4b8 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:51.384743 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 972/1907] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:44:51 2022] POST /identity/v3/projects => generated 399 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:44:51.385961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b6e21-c69b-4044-b790-21a9d3cec6d2 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:51.403154 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b6e21-c69b-4044-b790-21a9d3cec6d2 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:51.569859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b735d-3f62-4ede-b80a-4f317d30f306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:51.574181 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b735d-3f62-4ede-b80a-4f317d30f306 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:51.574783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b735d-3f62-4ede-b80a-4f317d30f306 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:51.575220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b735d-3f62-4ede-b80a-4f317d30f306 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:51.576009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b735d-3f62-4ede-b80a-4f317d30f306 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:51.578173 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b735d-3f62-4ede-b80a-4f317d30f306 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:51.732798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 936/1908] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:51 2022] POST /identity/v3/users => generated 306 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:51.736826 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8227d1-d0e7-4ed2-a419-10c5c0a9e4b8 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:51.747698 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8227d1-d0e7-4ed2-a419-10c5c0a9e4b8 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:51.752636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626a5d2-53b0-4842-9475-abb4a09602c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:51.753242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626a5d2-53b0-4842-9475-abb4a09602c1 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:51.756767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626a5d2-53b0-4842-9475-abb4a09602c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:51.756767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626a5d2-53b0-4842-9475-abb4a09602c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:51.756767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626a5d2-53b0-4842-9475-abb4a09602c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:51.756767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626a5d2-53b0-4842-9475-abb4a09602c1 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:51.768371 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 973/1909] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:51.773029 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b735d-3f62-4ede-b80a-4f317d30f306 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:51.796585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b735d-3f62-4ede-b80a-4f317d30f306 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:51.798785 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4facbe29-7f3d-4587-93fa-ef60e94f3643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:51.799467 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facbe29-7f3d-4587-93fa-ef60e94f3643 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:51.799751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facbe29-7f3d-4587-93fa-ef60e94f3643 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:51.800031 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facbe29-7f3d-4587-93fa-ef60e94f3643 admin admin] PATH_INFO: `/v3/projects/1345616642d14b2ea1b3a69ce7418fec/users/d944d2078e0b400da676b76a46c59a8c/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:51.819646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4facbe29-7f3d-4587-93fa-ef60e94f3643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1345616642d14b2ea1b3a69ce7418fec, user_id=d944d2078e0b400da676b76a46c59a8c, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:51.822407 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4facbe29-7f3d-4587-93fa-ef60e94f3643 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:51.935311 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 937/1910] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:51 2022] PUT /identity/v3/projects/1345616642d14b2ea1b3a69ce7418fec/users/d944d2078e0b400da676b76a46c59a8c/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:51.939329 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626a5d2-53b0-4842-9475-abb4a09602c1 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:51.950586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626a5d2-53b0-4842-9475-abb4a09602c1 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:52.021815 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d99887-7ed3-4250-a333-8a0960897f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:52.022462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d99887-7ed3-4250-a333-8a0960897f43 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:52.022915 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d99887-7ed3-4250-a333-8a0960897f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:52.023191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d99887-7ed3-4250-a333-8a0960897f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:52.023868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d99887-7ed3-4250-a333-8a0960897f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:52.025566 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d99887-7ed3-4250-a333-8a0960897f43 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:52.045917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4facbe29-7f3d-4587-93fa-ef60e94f3643 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:52.046747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 974/1911] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:52.058731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4facbe29-7f3d-4587-93fa-ef60e94f3643 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:52.064185 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a158f7-f710-4e66-a68b-b3f7b29c2e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:52.064875 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a158f7-f710-4e66-a68b-b3f7b29c2e1f admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:52.065153 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a158f7-f710-4e66-a68b-b3f7b29c2e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:52.065481 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a158f7-f710-4e66-a68b-b3f7b29c2e1f admin admin] PATH_INFO: `/v3/projects/1345616642d14b2ea1b3a69ce7418fec/users/d944d2078e0b400da676b76a46c59a8c/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:52.067628 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a158f7-f710-4e66-a68b-b3f7b29c2e1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1345616642d14b2ea1b3a69ce7418fec, user_id=d944d2078e0b400da676b76a46c59a8c, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:52.070535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a158f7-f710-4e66-a68b-b3f7b29c2e1f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:52.083149 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 938/1912] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:52 2022] PUT /identity/v3/projects/1345616642d14b2ea1b3a69ce7418fec/users/d944d2078e0b400da676b76a46c59a8c/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:44:52.093838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166bd7e-3e25-4a1b-97dc-9caf66a84b8c None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:52.094193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166bd7e-3e25-4a1b-97dc-9caf66a84b8c None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:52.098861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166bd7e-3e25-4a1b-97dc-9caf66a84b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:52.127423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-d166bd7e-3e25-4a1b-97dc-9caf66a84b8c None None] MFA Rules not processed for user `d944d2078e0b400da676b76a46c59a8c`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:52.129372 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-d166bd7e-3e25-4a1b-97dc-9caf66a84b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:52.205471 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 975/1913] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:52 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:52.323937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a158f7-f710-4e66-a68b-b3f7b29c2e1f admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:52.328175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-00a158f7-f710-4e66-a68b-b3f7b29c2e1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:52.418830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a158f7-f710-4e66-a68b-b3f7b29c2e1f admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:52.431308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b961eba-bd46-447d-a061-b1467a2ec074 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:52.432067 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b961eba-bd46-447d-a061-b1467a2ec074 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:52.432395 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b961eba-bd46-447d-a061-b1467a2ec074 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:52.432654 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b961eba-bd46-447d-a061-b1467a2ec074 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:52.449549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b961eba-bd46-447d-a061-b1467a2ec074 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:52.451001 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b961eba-bd46-447d-a061-b1467a2ec074 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:52.462349 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 939/1914] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:52 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:53.103045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166bd7e-3e25-4a1b-97dc-9caf66a84b8c None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:53.105571 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-d166bd7e-3e25-4a1b-97dc-9caf66a84b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:53.123670 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166bd7e-3e25-4a1b-97dc-9caf66a84b8c None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:53.127819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bbdbc8-2cd7-4339-be74-7c10897977ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:53.128579 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbdbc8-2cd7-4339-be74-7c10897977ab service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:53.128947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbdbc8-2cd7-4339-be74-7c10897977ab service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:53.129916 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbdbc8-2cd7-4339-be74-7c10897977ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:53.141968 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:53.201016 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bbdbc8-2cd7-4339-be74-7c10897977ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:53.203140 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bbdbc8-2cd7-4339-be74-7c10897977ab service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:53.216324 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 976/1915] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:53 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:54.206230 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:56.221345 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:56.400105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b961eba-bd46-447d-a061-b1467a2ec074 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:56.401542 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-8b961eba-bd46-447d-a061-b1467a2ec074 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:56.472001 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b961eba-bd46-447d-a061-b1467a2ec074 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:56.477733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c60a7f-1aee-42c9-8a36-d598b6b322cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:56.478400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c60a7f-1aee-42c9-8a36-d598b6b322cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:56.478675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c60a7f-1aee-42c9-8a36-d598b6b322cf admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:56.478942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c60a7f-1aee-42c9-8a36-d598b6b322cf admin admin] PATH_INFO: `/v3/users/a201508ed31d4a6f8095e26d40e04740` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:56.480099 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c60a7f-1aee-42c9-8a36-d598b6b322cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a201508ed31d4a6f8095e26d40e04740)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:56.482129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c60a7f-1aee-42c9-8a36-d598b6b322cf admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:56.831351 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-03c60a7f-1aee-42c9-8a36-d598b6b322cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a201508ed31d4a6f8095e26d40e04740'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:56.839340 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-03c60a7f-1aee-42c9-8a36-d598b6b322cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a201508ed31d4a6f8095e26d40e04740'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:56.840027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-03c60a7f-1aee-42c9-8a36-d598b6b322cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a201508ed31d4a6f8095e26d40e04740'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:56.852501 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-03c60a7f-1aee-42c9-8a36-d598b6b322cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a201508ed31d4a6f8095e26d40e04740'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:56.853178 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-03c60a7f-1aee-42c9-8a36-d598b6b322cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a201508ed31d4a6f8095e26d40e04740'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:56.963016 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 940/1916] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:44:56 2022] DELETE /identity/v3/users/a201508ed31d4a6f8095e26d40e04740 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:56.964642 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce1d75-f920-401a-947a-245a0fcb8aca None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:56.965070 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce1d75-f920-401a-947a-245a0fcb8aca None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:56.965186 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce1d75-f920-401a-947a-245a0fcb8aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:56.984909 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-38ce1d75-f920-401a-947a-245a0fcb8aca None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:56.986115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-38ce1d75-f920-401a-947a-245a0fcb8aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:57.046499 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 977/1917] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:57.057340 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c60a7f-1aee-42c9-8a36-d598b6b322cf admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:57.058761 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-03c60a7f-1aee-42c9-8a36-d598b6b322cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:57.100953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:57.120782 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c60a7f-1aee-42c9-8a36-d598b6b322cf admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:57.124657 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50e08b-eb0b-4214-a138-7a8a2c29642d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:57.134307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50e08b-eb0b-4214-a138-7a8a2c29642d service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:57.134307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50e08b-eb0b-4214-a138-7a8a2c29642d service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:57.134307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50e08b-eb0b-4214-a138-7a8a2c29642d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:57.136434 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50e08b-eb0b-4214-a138-7a8a2c29642d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:57.137917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50e08b-eb0b-4214-a138-7a8a2c29642d service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:57.148701 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 941/1918] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:44:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:44:57.692481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce1d75-f920-401a-947a-245a0fcb8aca None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:57.693936 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-38ce1d75-f920-401a-947a-245a0fcb8aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:57.707650 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce1d75-f920-401a-947a-245a0fcb8aca None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:57.713255 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57259dd-be5e-4028-acc2-d06f74a4c63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:57.713913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57259dd-be5e-4028-acc2-d06f74a4c63b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:57.714193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57259dd-be5e-4028-acc2-d06f74a4c63b admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:57.714495 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57259dd-be5e-4028-acc2-d06f74a4c63b admin admin] PATH_INFO: `/v3/projects/778425318e494e6a9e054b18a2f22e16` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:57.715677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57259dd-be5e-4028-acc2-d06f74a4c63b admin admin] RBAC: Authorizing `identity:delete_project(project_id=778425318e494e6a9e054b18a2f22e16)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:57.717617 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57259dd-be5e-4028-acc2-d06f74a4c63b admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:57.853340 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:57.863084 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-e57259dd-be5e-4028-acc2-d06f74a4c63b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '778425318e494e6a9e054b18a2f22e16', 'request_id': 'req-e57259dd-be5e-4028-acc2-d06f74a4c63b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:57.869715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-e57259dd-be5e-4028-acc2-d06f74a4c63b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '778425318e494e6a9e054b18a2f22e16', 'request_id': 'req-e57259dd-be5e-4028-acc2-d06f74a4c63b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:57.875387 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-e57259dd-be5e-4028-acc2-d06f74a4c63b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '778425318e494e6a9e054b18a2f22e16', 'request_id': 'req-e57259dd-be5e-4028-acc2-d06f74a4c63b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:57.969634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-e57259dd-be5e-4028-acc2-d06f74a4c63b admin admin] The token cache is being invalidate because project 778425318e494e6a9e054b18a2f22e16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:44:57.969634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-e57259dd-be5e-4028-acc2-d06f74a4c63b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:57.970435 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-e57259dd-be5e-4028-acc2-d06f74a4c63b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:44:57.971707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 978/1919] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:44:57 2022] DELETE /identity/v3/projects/778425318e494e6a9e054b18a2f22e16 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:57.977584 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38180c4-6541-4bd9-9807-3264798d9908 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:57.977993 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38180c4-6541-4bd9-9807-3264798d9908 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:57.978121 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38180c4-6541-4bd9-9807-3264798d9908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:57.998406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-f38180c4-6541-4bd9-9807-3264798d9908 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:57.999598 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f38180c4-6541-4bd9-9807-3264798d9908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:58.070619 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 942/1920] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:58.079299 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce03362-5714-4842-bc0e-f90742c2298e None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:58.079581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce03362-5714-4842-bc0e-f90742c2298e None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:58.079856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce03362-5714-4842-bc0e-f90742c2298e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:58.099773 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-4ce03362-5714-4842-bc0e-f90742c2298e None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:58.101696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-4ce03362-5714-4842-bc0e-f90742c2298e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:58.109672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 979/1921] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:58.114013 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38180c4-6541-4bd9-9807-3264798d9908 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:58.125018 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38180c4-6541-4bd9-9807-3264798d9908 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:58.129666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f984b-a680-4e56-9a54-e05295f0370e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:58.130280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f984b-a680-4e56-9a54-e05295f0370e admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:58.130559 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f984b-a680-4e56-9a54-e05295f0370e admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:58.130833 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f984b-a680-4e56-9a54-e05295f0370e admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:58.131484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f984b-a680-4e56-9a54-e05295f0370e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:44:58.131808 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f984b-a680-4e56-9a54-e05295f0370e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:58.133493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f984b-a680-4e56-9a54-e05295f0370e admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:58.146189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 943/1922] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:44:58.150666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce03362-5714-4842-bc0e-f90742c2298e None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:58.160056 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce03362-5714-4842-bc0e-f90742c2298e None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:58.165333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f7bc5-115a-4022-829f-97b579ccad50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:58.165932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f7bc5-115a-4022-829f-97b579ccad50 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:58.166210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f7bc5-115a-4022-829f-97b579ccad50 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:58.166483 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f7bc5-115a-4022-829f-97b579ccad50 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:58.167120 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f7bc5-115a-4022-829f-97b579ccad50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:58.169093 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f7bc5-115a-4022-829f-97b579ccad50 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:58.261668 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 980/1923] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:44:58 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:44:58.273057 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f984b-a680-4e56-9a54-e05295f0370e admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:58.283081 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f984b-a680-4e56-9a54-e05295f0370e admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:58.346424 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc5093-bcf1-4d52-8bf4-a4e0a142982a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:58.347036 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc5093-bcf1-4d52-8bf4-a4e0a142982a admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:58.347314 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc5093-bcf1-4d52-8bf4-a4e0a142982a admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:58.347585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc5093-bcf1-4d52-8bf4-a4e0a142982a admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:58.348269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bc5093-bcf1-4d52-8bf4-a4e0a142982a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:58.350196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bc5093-bcf1-4d52-8bf4-a4e0a142982a admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:58.501666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 944/1924] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:58 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:58.508653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f7bc5-115a-4022-829f-97b579ccad50 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:58.519112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f7bc5-115a-4022-829f-97b579ccad50 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:58.524674 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21778460-92f2-416f-97c3-e7ee165adbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:58.525194 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21778460-92f2-416f-97c3-e7ee165adbf6 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:58.525474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21778460-92f2-416f-97c3-e7ee165adbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:58.525752 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21778460-92f2-416f-97c3-e7ee165adbf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:58.526398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21778460-92f2-416f-97c3-e7ee165adbf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:58.528035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21778460-92f2-416f-97c3-e7ee165adbf6 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:58.539515 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 981/1925] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:44:58.543746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc5093-bcf1-4d52-8bf4-a4e0a142982a admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:58.554081 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc5093-bcf1-4d52-8bf4-a4e0a142982a admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:58.558926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cce7c-bfd3-4faf-b16f-23062a74c910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:58.559524 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cce7c-bfd3-4faf-b16f-23062a74c910 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:58.559799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cce7c-bfd3-4faf-b16f-23062a74c910 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:58.560091 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cce7c-bfd3-4faf-b16f-23062a74c910 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:58.560756 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2cce7c-bfd3-4faf-b16f-23062a74c910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:58.562391 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2cce7c-bfd3-4faf-b16f-23062a74c910 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:58.573876 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 945/1926] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:58.578049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21778460-92f2-416f-97c3-e7ee165adbf6 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:58.589340 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21778460-92f2-416f-97c3-e7ee165adbf6 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:58.593986 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2132d57-9661-4da6-9c03-248801a4ca28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:58.594660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2132d57-9661-4da6-9c03-248801a4ca28 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:58.596574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2132d57-9661-4da6-9c03-248801a4ca28 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:58.596574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2132d57-9661-4da6-9c03-248801a4ca28 admin admin] PATH_INFO: `/v3/projects/dd4cbefa02e8484e8f0e9438f96a19f1/users/5b560adb99bc4c408d23fe84a8709a9b/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:58.610833 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2132d57-9661-4da6-9c03-248801a4ca28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd4cbefa02e8484e8f0e9438f96a19f1, user_id=5b560adb99bc4c408d23fe84a8709a9b, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:58.613589 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2132d57-9661-4da6-9c03-248801a4ca28 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:58.717103 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 982/1927] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:58 2022] PUT /identity/v3/projects/dd4cbefa02e8484e8f0e9438f96a19f1/users/5b560adb99bc4c408d23fe84a8709a9b/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:58.724921 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ef698-5d24-4ceb-a3fe-8696d2face86 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:58.726013 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ef698-5d24-4ceb-a3fe-8696d2face86 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:58.726369 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ef698-5d24-4ceb-a3fe-8696d2face86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:58.756008 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-236ef698-5d24-4ceb-a3fe-8696d2face86 None None] MFA Rules not processed for user `5b560adb99bc4c408d23fe84a8709a9b`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:58.757320 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-236ef698-5d24-4ceb-a3fe-8696d2face86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:58.786974 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:58.822546 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 946/1928] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:44:58.827126 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2132d57-9661-4da6-9c03-248801a4ca28 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:58.891737 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2132d57-9661-4da6-9c03-248801a4ca28 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:58.896716 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816881c2-a7ad-4d67-af9e-a4f5e219d007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:58.898097 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816881c2-a7ad-4d67-af9e-a4f5e219d007 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:58.900440 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816881c2-a7ad-4d67-af9e-a4f5e219d007 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:58.900440 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816881c2-a7ad-4d67-af9e-a4f5e219d007 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:58.900440 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816881c2-a7ad-4d67-af9e-a4f5e219d007 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:58.901436 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816881c2-a7ad-4d67-af9e-a4f5e219d007 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:59.000031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 983/1929] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:44:58 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:59.004349 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ef698-5d24-4ceb-a3fe-8696d2face86 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:59.016634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ef698-5d24-4ceb-a3fe-8696d2face86 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:59.078374 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a754ae-cd08-48e3-9a37-a80433d7e72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:59.079054 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a754ae-cd08-48e3-9a37-a80433d7e72d admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:59.079360 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a754ae-cd08-48e3-9a37-a80433d7e72d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:59.079633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a754ae-cd08-48e3-9a37-a80433d7e72d admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:59.080300 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a754ae-cd08-48e3-9a37-a80433d7e72d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:59.082967 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a754ae-cd08-48e3-9a37-a80433d7e72d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:59.170422 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 947/1930] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:59 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:59.174863 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816881c2-a7ad-4d67-af9e-a4f5e219d007 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:59.186170 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816881c2-a7ad-4d67-af9e-a4f5e219d007 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:59.191011 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233d0de-76e3-455f-bc60-cb565b2c8b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:59.191601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233d0de-76e3-455f-bc60-cb565b2c8b7f admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:59.191877 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233d0de-76e3-455f-bc60-cb565b2c8b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:59.192152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233d0de-76e3-455f-bc60-cb565b2c8b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:59.192800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233d0de-76e3-455f-bc60-cb565b2c8b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:59.194389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233d0de-76e3-455f-bc60-cb565b2c8b7f admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:59.206558 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 984/1931] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:59.211544 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a754ae-cd08-48e3-9a37-a80433d7e72d admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:59.221843 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a754ae-cd08-48e3-9a37-a80433d7e72d admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:59.226447 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b689d51d-54be-4290-9d63-e8de1afbf5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:59.227689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689d51d-54be-4290-9d63-e8de1afbf5cb admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:59.227966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689d51d-54be-4290-9d63-e8de1afbf5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:59.228266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689d51d-54be-4290-9d63-e8de1afbf5cb admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:59.228921 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b689d51d-54be-4290-9d63-e8de1afbf5cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:59.230493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b689d51d-54be-4290-9d63-e8de1afbf5cb admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:59.240629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 948/1932] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:44:59.245115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233d0de-76e3-455f-bc60-cb565b2c8b7f admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:59.255778 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233d0de-76e3-455f-bc60-cb565b2c8b7f admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:59.261111 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7daf37-eedb-430a-bd04-cf7f009fa1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:59.261931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7daf37-eedb-430a-bd04-cf7f009fa1c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:59.262250 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7daf37-eedb-430a-bd04-cf7f009fa1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:59.262574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7daf37-eedb-430a-bd04-cf7f009fa1c5 admin admin] PATH_INFO: `/v3/projects/be48a4aa8fcd427f99d5d66c48c722da/users/16a7cd241f50498fa0d334e163d7792a/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:59.280470 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7daf37-eedb-430a-bd04-cf7f009fa1c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be48a4aa8fcd427f99d5d66c48c722da, user_id=16a7cd241f50498fa0d334e163d7792a, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:59.283301 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7daf37-eedb-430a-bd04-cf7f009fa1c5 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:59.399121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 985/1933] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:44:59 2022] PUT /identity/v3/projects/be48a4aa8fcd427f99d5d66c48c722da/users/16a7cd241f50498fa0d334e163d7792a/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:44:59.415493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69df1d-16ba-480f-a2c4-035fc7b270c5 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:59.415870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69df1d-16ba-480f-a2c4-035fc7b270c5 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:59.416235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69df1d-16ba-480f-a2c4-035fc7b270c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:59.454130 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-1b69df1d-16ba-480f-a2c4-035fc7b270c5 None None] MFA Rules not processed for user `16a7cd241f50498fa0d334e163d7792a`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:44:59.455448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-1b69df1d-16ba-480f-a2c4-035fc7b270c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:44:59.531650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 949/1934] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:44:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:44:59.536302 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7daf37-eedb-430a-bd04-cf7f009fa1c5 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:59.548901 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:59.610633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7daf37-eedb-430a-bd04-cf7f009fa1c5 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:59.617212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff195f-698e-4533-b333-136835ccdaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:59.617896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff195f-698e-4533-b333-136835ccdaf1 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:59.618171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff195f-698e-4533-b333-136835ccdaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:59.618554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff195f-698e-4533-b333-136835ccdaf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:59.619205 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff195f-698e-4533-b333-136835ccdaf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:59.621311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff195f-698e-4533-b333-136835ccdaf1 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:59.740050 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 986/1935] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:44:59 2022] POST /identity/v3/projects => generated 377 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:59.741406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69df1d-16ba-480f-a2c4-035fc7b270c5 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:59.753402 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69df1d-16ba-480f-a2c4-035fc7b270c5 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:59.799719 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:44:59.825562 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567d23d-c252-428f-94e3-cbb06a3468e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:59.826173 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567d23d-c252-428f-94e3-cbb06a3468e1 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:59.826446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567d23d-c252-428f-94e3-cbb06a3468e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:59.826711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567d23d-c252-428f-94e3-cbb06a3468e1 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:59.827336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a567d23d-c252-428f-94e3-cbb06a3468e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:59.829271 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a567d23d-c252-428f-94e3-cbb06a3468e1 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:44:59.967539 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 950/1936] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:44:59 2022] POST /identity/v3/users => generated 295 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:44:59.970909 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff195f-698e-4533-b333-136835ccdaf1 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:44:59.982474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff195f-698e-4533-b333-136835ccdaf1 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:44:59.987179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcdbe9-76e0-4814-97b4-571beeaa5589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:44:59.987785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcdbe9-76e0-4814-97b4-571beeaa5589 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:44:59.988059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcdbe9-76e0-4814-97b4-571beeaa5589 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:44:59.988354 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcdbe9-76e0-4814-97b4-571beeaa5589 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:44:59.988996 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bcdbe9-76e0-4814-97b4-571beeaa5589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:44:59.990567 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bcdbe9-76e0-4814-97b4-571beeaa5589 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:00.006444 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 987/1937] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:00.007632 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567d23d-c252-428f-94e3-cbb06a3468e1 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:00.020413 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567d23d-c252-428f-94e3-cbb06a3468e1 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:00.026566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b0e74-fbc7-4cd2-b866-0892b4bd6971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:00.027246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b0e74-fbc7-4cd2-b866-0892b4bd6971 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:00.031047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b0e74-fbc7-4cd2-b866-0892b4bd6971 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:00.031047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b0e74-fbc7-4cd2-b866-0892b4bd6971 admin admin] PATH_INFO: `/v3/projects/2c2b0032de4c436b95efa8a3c1af8491/users/f5dad0204ef147d18cbeba5b7cdd7f0a/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:00.044326 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b0e74-fbc7-4cd2-b866-0892b4bd6971 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c2b0032de4c436b95efa8a3c1af8491, user_id=f5dad0204ef147d18cbeba5b7cdd7f0a, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:00.047010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b0e74-fbc7-4cd2-b866-0892b4bd6971 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:00.131005 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcdbe9-76e0-4814-97b4-571beeaa5589 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:00.132787 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-28bcdbe9-76e0-4814-97b4-571beeaa5589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:00.147780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcdbe9-76e0-4814-97b4-571beeaa5589 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:00.158578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 951/1938] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:00 2022] PUT /identity/v3/projects/2c2b0032de4c436b95efa8a3c1af8491/users/f5dad0204ef147d18cbeba5b7cdd7f0a/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:45:00.159587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25846b4-6b2e-4ec4-a62c-601a92e7eea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:00.160262 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25846b4-6b2e-4ec4-a62c-601a92e7eea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:00.160537 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25846b4-6b2e-4ec4-a62c-601a92e7eea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:00.160969 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25846b4-6b2e-4ec4-a62c-601a92e7eea7 admin admin] PATH_INFO: `/v3/users/1fa4f486d0c14d5cba39035a5c7985b6` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:00.161849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b0e74-fbc7-4cd2-b866-0892b4bd6971 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:00.161955 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25846b4-6b2e-4ec4-a62c-601a92e7eea7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fa4f486d0c14d5cba39035a5c7985b6)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:00.164670 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25846b4-6b2e-4ec4-a62c-601a92e7eea7 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:00.238222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b0e74-fbc7-4cd2-b866-0892b4bd6971 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:00.244400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0042b-1fbe-4803-8b63-ea99e9d245ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:00.244661 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0042b-1fbe-4803-8b63-ea99e9d245ce admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:00.244942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0042b-1fbe-4803-8b63-ea99e9d245ce admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:00.245212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0042b-1fbe-4803-8b63-ea99e9d245ce admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:00.245890 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0042b-1fbe-4803-8b63-ea99e9d245ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:00.247522 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0042b-1fbe-4803-8b63-ea99e9d245ce admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:00.259766 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 952/1939] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:45:00.264068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0042b-1fbe-4803-8b63-ea99e9d245ce admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:00.275572 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0042b-1fbe-4803-8b63-ea99e9d245ce admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:00.283086 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55dfe8-4030-47b7-8907-13267c77bb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:00.283915 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55dfe8-4030-47b7-8907-13267c77bb8e admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:00.284336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55dfe8-4030-47b7-8907-13267c77bb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:00.285377 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55dfe8-4030-47b7-8907-13267c77bb8e admin admin] PATH_INFO: `/v3/projects/2c2b0032de4c436b95efa8a3c1af8491/users/f5dad0204ef147d18cbeba5b7cdd7f0a/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:00.287621 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55dfe8-4030-47b7-8907-13267c77bb8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c2b0032de4c436b95efa8a3c1af8491, user_id=f5dad0204ef147d18cbeba5b7cdd7f0a, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:00.290746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55dfe8-4030-47b7-8907-13267c77bb8e admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:00.308177 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 953/1940] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:00 2022] PUT /identity/v3/projects/2c2b0032de4c436b95efa8a3c1af8491/users/f5dad0204ef147d18cbeba5b7cdd7f0a/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:45:00.314153 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4a74e-9942-4cf7-91e7-5a97ff595a8c None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:00.318199 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4a74e-9942-4cf7-91e7-5a97ff595a8c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:00.318199 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4a74e-9942-4cf7-91e7-5a97ff595a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:00.359008 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-bde4a74e-9942-4cf7-91e7-5a97ff595a8c None None] MFA Rules not processed for user `f5dad0204ef147d18cbeba5b7cdd7f0a`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:00.360800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-bde4a74e-9942-4cf7-91e7-5a97ff595a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:00.393959 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b25846b4-6b2e-4ec4-a62c-601a92e7eea7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fa4f486d0c14d5cba39035a5c7985b6'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:00.395044 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b25846b4-6b2e-4ec4-a62c-601a92e7eea7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fa4f486d0c14d5cba39035a5c7985b6'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:00.402932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b25846b4-6b2e-4ec4-a62c-601a92e7eea7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fa4f486d0c14d5cba39035a5c7985b6'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:00.443038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 954/1941] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:45:00.454637 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4a74e-9942-4cf7-91e7-5a97ff595a8c None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:00.456996 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-bde4a74e-9942-4cf7-91e7-5a97ff595a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:00.471121 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:00.478947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b25846b4-6b2e-4ec4-a62c-601a92e7eea7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fa4f486d0c14d5cba39035a5c7985b6'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:00.480087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b25846b4-6b2e-4ec4-a62c-601a92e7eea7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fa4f486d0c14d5cba39035a5c7985b6'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:00.500033 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 988/1942] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:45:00 2022] DELETE /identity/v3/users/1fa4f486d0c14d5cba39035a5c7985b6 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:45:00.504493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f880b4-4f9d-4765-9b95-0edca25ab0c0 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:00.504773 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f880b4-4f9d-4765-9b95-0edca25ab0c0 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:00.505043 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f880b4-4f9d-4765-9b95-0edca25ab0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:00.520000 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4a74e-9942-4cf7-91e7-5a97ff595a8c None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:00.524851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-60f880b4-4f9d-4765-9b95-0edca25ab0c0 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:00.526058 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-60f880b4-4f9d-4765-9b95-0edca25ab0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:00.571781 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab791a8-9986-4ed3-a550-b0b208147cf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:00.572443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab791a8-9986-4ed3-a550-b0b208147cf3 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:00.572721 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab791a8-9986-4ed3-a550-b0b208147cf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:00.572983 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab791a8-9986-4ed3-a550-b0b208147cf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:00.574780 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-dab791a8-9986-4ed3-a550-b0b208147cf3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:00.579599 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:00.591420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 989/1943] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:45:00.612672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f880b4-4f9d-4765-9b95-0edca25ab0c0 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:00.615147 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-60f880b4-4f9d-4765-9b95-0edca25ab0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:00.652722 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab791a8-9986-4ed3-a550-b0b208147cf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:00.654172 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab791a8-9986-4ed3-a550-b0b208147cf3 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:00.665021 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 955/1944] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:45:00.694050 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f880b4-4f9d-4765-9b95-0edca25ab0c0 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:00.694050 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943d16a-e0b7-45ad-8fc4-de97acc03b79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:00.694965 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943d16a-e0b7-45ad-8fc4-de97acc03b79 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:00.695451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943d16a-e0b7-45ad-8fc4-de97acc03b79 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:00.696329 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943d16a-e0b7-45ad-8fc4-de97acc03b79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:00.709726 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1943d16a-e0b7-45ad-8fc4-de97acc03b79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:00.713025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1943d16a-e0b7-45ad-8fc4-de97acc03b79 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:00.725601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 990/1945] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:45:01.172266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083c59b-d738-4f28-93f6-4e106398714d None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:01.172863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083c59b-d738-4f28-93f6-4e106398714d None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:01.172863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083c59b-d738-4f28-93f6-4e106398714d None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:01.174171 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 956/1946] 10.222.0.59 () {54 vars in 921 bytes} [Mon Aug 29 22:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:45:01.429188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e6c6d-28fe-4a5b-9688-7e6a15c85a68 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:01.429188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e6c6d-28fe-4a5b-9688-7e6a15c85a68 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:01.429188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e6c6d-28fe-4a5b-9688-7e6a15c85a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:01.463572 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-859e6c6d-28fe-4a5b-9688-7e6a15c85a68 None None] MFA Rules not processed for user `7215ee41033742d79671f4f42b81cd06`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:01.464856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-859e6c6d-28fe-4a5b-9688-7e6a15c85a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:01.535854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 991/1947] 10.222.0.59 () {58 vars in 1018 bytes} [Mon Aug 29 22:45:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:01.547981 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083c59b-d738-4f28-93f6-4e106398714d None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:01.549698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-5083c59b-d738-4f28-93f6-4e106398714d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:01.567127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083c59b-d738-4f28-93f6-4e106398714d None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:01.573348 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df973af9-f660-4a05-b3ce-9d77fc278e03 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:01.574067 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df973af9-f660-4a05-b3ce-9d77fc278e03 service placement] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:01.574310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df973af9-f660-4a05-b3ce-9d77fc278e03 service placement] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:01.574652 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df973af9-f660-4a05-b3ce-9d77fc278e03 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:01.590389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df973af9-f660-4a05-b3ce-9d77fc278e03 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:01.595321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df973af9-f660-4a05-b3ce-9d77fc278e03 service placement] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:01.612973 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 957/1948] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:01.893635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e6c6d-28fe-4a5b-9688-7e6a15c85a68 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:01.895511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-859e6c6d-28fe-4a5b-9688-7e6a15c85a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:01.918623 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e6c6d-28fe-4a5b-9688-7e6a15c85a68 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:01.928458 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58c763-7d61-40cc-8af4-eefa5d92912e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:01.929435 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58c763-7d61-40cc-8af4-eefa5d92912e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:01.930860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58c763-7d61-40cc-8af4-eefa5d92912e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:01.931534 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58c763-7d61-40cc-8af4-eefa5d92912e admin admin] PATH_INFO: `/v3/projects/7900f30129fa4a5bb015becb1149b4e4` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:01.933010 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58c763-7d61-40cc-8af4-eefa5d92912e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7900f30129fa4a5bb015becb1149b4e4)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:01.935296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58c763-7d61-40cc-8af4-eefa5d92912e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:02.074755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6f58c763-7d61-40cc-8af4-eefa5d92912e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7900f30129fa4a5bb015becb1149b4e4', 'request_id': 'req-6f58c763-7d61-40cc-8af4-eefa5d92912e'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:02.081141 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6f58c763-7d61-40cc-8af4-eefa5d92912e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7900f30129fa4a5bb015becb1149b4e4', 'request_id': 'req-6f58c763-7d61-40cc-8af4-eefa5d92912e'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:02.087651 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6f58c763-7d61-40cc-8af4-eefa5d92912e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7900f30129fa4a5bb015becb1149b4e4', 'request_id': 'req-6f58c763-7d61-40cc-8af4-eefa5d92912e'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:02.173214 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6f58c763-7d61-40cc-8af4-eefa5d92912e admin admin] The token cache is being invalidate because project 7900f30129fa4a5bb015becb1149b4e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:45:02.173214 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6f58c763-7d61-40cc-8af4-eefa5d92912e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:02.173214 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6f58c763-7d61-40cc-8af4-eefa5d92912e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:02.174196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 992/1949] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:45:01 2022] DELETE /identity/v3/projects/7900f30129fa4a5bb015becb1149b4e4 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:02.177963 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df973af9-f660-4a05-b3ce-9d77fc278e03 service placement] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:02.190066 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df973af9-f660-4a05-b3ce-9d77fc278e03 service placement] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:02.253994 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781dfed0-24fe-4583-b172-f46acf55a15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:02.254650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781dfed0-24fe-4583-b172-f46acf55a15c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:02.257990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781dfed0-24fe-4583-b172-f46acf55a15c admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:02.257990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781dfed0-24fe-4583-b172-f46acf55a15c admin admin] PATH_INFO: `/v3/users/d944d2078e0b400da676b76a46c59a8c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:02.257990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781dfed0-24fe-4583-b172-f46acf55a15c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d944d2078e0b400da676b76a46c59a8c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:02.258734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781dfed0-24fe-4583-b172-f46acf55a15c admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:02.269796 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:02.374949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58c763-7d61-40cc-8af4-eefa5d92912e admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:02.377422 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6f58c763-7d61-40cc-8af4-eefa5d92912e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:02.402686 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58c763-7d61-40cc-8af4-eefa5d92912e admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:02.408653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b017f2b-5f50-4255-8601-86c1c84dcc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:02.409271 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b017f2b-5f50-4255-8601-86c1c84dcc97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:02.409549 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b017f2b-5f50-4255-8601-86c1c84dcc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:02.409818 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b017f2b-5f50-4255-8601-86c1c84dcc97 admin admin] PATH_INFO: `/v3/users/1102f5f98bf545ada39c019f45f2f413` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:02.410942 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b017f2b-5f50-4255-8601-86c1c84dcc97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1102f5f98bf545ada39c019f45f2f413)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:02.412897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b017f2b-5f50-4255-8601-86c1c84dcc97 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:02.427754 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:02.482468 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-781dfed0-24fe-4583-b172-f46acf55a15c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd944d2078e0b400da676b76a46c59a8c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:02.491046 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-781dfed0-24fe-4583-b172-f46acf55a15c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd944d2078e0b400da676b76a46c59a8c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:02.491455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-781dfed0-24fe-4583-b172-f46acf55a15c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd944d2078e0b400da676b76a46c59a8c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:02.505905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-781dfed0-24fe-4583-b172-f46acf55a15c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd944d2078e0b400da676b76a46c59a8c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:02.506756 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-781dfed0-24fe-4583-b172-f46acf55a15c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd944d2078e0b400da676b76a46c59a8c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:02.639041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 958/1950] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:45:02 2022] DELETE /identity/v3/users/d944d2078e0b400da676b76a46c59a8c => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:02.762173 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8b017f2b-5f50-4255-8601-86c1c84dcc97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1102f5f98bf545ada39c019f45f2f413'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:02.762858 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8b017f2b-5f50-4255-8601-86c1c84dcc97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1102f5f98bf545ada39c019f45f2f413'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:02.763127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781dfed0-24fe-4583-b172-f46acf55a15c admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:02.764651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-781dfed0-24fe-4583-b172-f46acf55a15c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:02.769888 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8b017f2b-5f50-4255-8601-86c1c84dcc97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1102f5f98bf545ada39c019f45f2f413'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:02.825009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781dfed0-24fe-4583-b172-f46acf55a15c admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:02.828811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b9893-2284-4ea5-b495-65d321624e85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:02.829426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b9893-2284-4ea5-b495-65d321624e85 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:02.829704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b9893-2284-4ea5-b495-65d321624e85 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:02.829978 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b9893-2284-4ea5-b495-65d321624e85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:02.831711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-c69b9893-2284-4ea5-b495-65d321624e85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:02.889877 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8b017f2b-5f50-4255-8601-86c1c84dcc97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1102f5f98bf545ada39c019f45f2f413'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:02.890691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8b017f2b-5f50-4255-8601-86c1c84dcc97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1102f5f98bf545ada39c019f45f2f413'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:02.901326 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b9893-2284-4ea5-b495-65d321624e85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:02.902809 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b9893-2284-4ea5-b495-65d321624e85 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:02.914482 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 993/1951] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:45:02 2022] DELETE /identity/v3/users/1102f5f98bf545ada39c019f45f2f413 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:02.920710 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 959/1952] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:45:03.157457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b017f2b-5f50-4255-8601-86c1c84dcc97 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:03.158867 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8b017f2b-5f50-4255-8601-86c1c84dcc97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:03.212024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b9893-2284-4ea5-b495-65d321624e85 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:03.213742 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-c69b9893-2284-4ea5-b495-65d321624e85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:03.257702 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b017f2b-5f50-4255-8601-86c1c84dcc97 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:03.263748 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4f15b-1fb1-4a08-b091-6e8767c236e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:03.264767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4f15b-1fb1-4a08-b091-6e8767c236e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:03.265215 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4f15b-1fb1-4a08-b091-6e8767c236e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:03.265764 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4f15b-1fb1-4a08-b091-6e8767c236e6 admin admin] PATH_INFO: `/v3/projects/1345616642d14b2ea1b3a69ce7418fec` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:03.267257 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4f15b-1fb1-4a08-b091-6e8767c236e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1345616642d14b2ea1b3a69ce7418fec)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:03.269469 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4f15b-1fb1-4a08-b091-6e8767c236e6 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:03.277663 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:03.303217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b9893-2284-4ea5-b495-65d321624e85 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:03.308811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44c8c2-59af-4653-9382-5ad8e2b3ba61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:03.309448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44c8c2-59af-4653-9382-5ad8e2b3ba61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:03.309799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44c8c2-59af-4653-9382-5ad8e2b3ba61 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:03.309991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44c8c2-59af-4653-9382-5ad8e2b3ba61 admin admin] PATH_INFO: `/v3/projects/40b5a74a98374cf999102f7cf16957d7` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:03.311298 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44c8c2-59af-4653-9382-5ad8e2b3ba61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40b5a74a98374cf999102f7cf16957d7)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:03.313370 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44c8c2-59af-4653-9382-5ad8e2b3ba61 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:03.436641 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-47b4f15b-1fb1-4a08-b091-6e8767c236e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1345616642d14b2ea1b3a69ce7418fec', 'request_id': 'req-47b4f15b-1fb1-4a08-b091-6e8767c236e6'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:03.443100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-47b4f15b-1fb1-4a08-b091-6e8767c236e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1345616642d14b2ea1b3a69ce7418fec', 'request_id': 'req-47b4f15b-1fb1-4a08-b091-6e8767c236e6'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:03.445629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-ce44c8c2-59af-4653-9382-5ad8e2b3ba61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40b5a74a98374cf999102f7cf16957d7', 'request_id': 'req-ce44c8c2-59af-4653-9382-5ad8e2b3ba61'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:03.449261 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-47b4f15b-1fb1-4a08-b091-6e8767c236e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1345616642d14b2ea1b3a69ce7418fec', 'request_id': 'req-47b4f15b-1fb1-4a08-b091-6e8767c236e6'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:03.517977 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-ce44c8c2-59af-4653-9382-5ad8e2b3ba61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40b5a74a98374cf999102f7cf16957d7', 'request_id': 'req-ce44c8c2-59af-4653-9382-5ad8e2b3ba61'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:03.525206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-ce44c8c2-59af-4653-9382-5ad8e2b3ba61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40b5a74a98374cf999102f7cf16957d7', 'request_id': 'req-ce44c8c2-59af-4653-9382-5ad8e2b3ba61'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:03.540600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-ce44c8c2-59af-4653-9382-5ad8e2b3ba61 admin admin] The token cache is being invalidate because project 40b5a74a98374cf999102f7cf16957d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:45:03.540992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-ce44c8c2-59af-4653-9382-5ad8e2b3ba61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:03.541861 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-ce44c8c2-59af-4653-9382-5ad8e2b3ba61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:03.547946 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 960/1953] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:45:03 2022] DELETE /identity/v3/projects/40b5a74a98374cf999102f7cf16957d7 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:45:03.550099 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44c8c2-59af-4653-9382-5ad8e2b3ba61 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:03.551016 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ce44c8c2-59af-4653-9382-5ad8e2b3ba61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:03.609594 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-47b4f15b-1fb1-4a08-b091-6e8767c236e6 admin admin] The token cache is being invalidate because project 1345616642d14b2ea1b3a69ce7418fec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:45:03.609886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-47b4f15b-1fb1-4a08-b091-6e8767c236e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:03.610505 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-47b4f15b-1fb1-4a08-b091-6e8767c236e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:03.612402 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 994/1954] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:45:03 2022] DELETE /identity/v3/projects/1345616642d14b2ea1b3a69ce7418fec => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:03.616993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e22ffe-f47a-4d9e-88c5-39e6cf7cf034 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:03.617886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e22ffe-f47a-4d9e-88c5-39e6cf7cf034 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:03.618235 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e22ffe-f47a-4d9e-88c5-39e6cf7cf034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:03.624950 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44c8c2-59af-4653-9382-5ad8e2b3ba61 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:03.631942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704c69c-2045-4254-a910-453f1a45b78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:03.632882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704c69c-2045-4254-a910-453f1a45b78f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:03.633562 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704c69c-2045-4254-a910-453f1a45b78f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:03.634222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704c69c-2045-4254-a910-453f1a45b78f admin admin] PATH_INFO: `/v3/users/fb6d101335a046498de9cda35ab3bc03` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:03.635797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704c69c-2045-4254-a910-453f1a45b78f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb6d101335a046498de9cda35ab3bc03)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:03.637905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704c69c-2045-4254-a910-453f1a45b78f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:03.639832 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-d7e22ffe-f47a-4d9e-88c5-39e6cf7cf034 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:03.641202 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-d7e22ffe-f47a-4d9e-88c5-39e6cf7cf034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:03.658058 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 995/1955] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:45:03.662110 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd087d4-2a1a-4f2f-91dc-d2ba597476e6 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:03.662577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd087d4-2a1a-4f2f-91dc-d2ba597476e6 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:03.670221 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd087d4-2a1a-4f2f-91dc-d2ba597476e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:03.684586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-2fd087d4-2a1a-4f2f-91dc-d2ba597476e6 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:03.685769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-2fd087d4-2a1a-4f2f-91dc-d2ba597476e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:03.693736 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 996/1956] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:03.698364 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd087d4-2a1a-4f2f-91dc-d2ba597476e6 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:03.707859 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd087d4-2a1a-4f2f-91dc-d2ba597476e6 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:03.712924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2444ecf-961d-425e-9ef9-36c1f24187e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:03.713511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2444ecf-961d-425e-9ef9-36c1f24187e0 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:03.713784 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2444ecf-961d-425e-9ef9-36c1f24187e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:03.714128 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2444ecf-961d-425e-9ef9-36c1f24187e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:03.714812 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2444ecf-961d-425e-9ef9-36c1f24187e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:45:03.715086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2444ecf-961d-425e-9ef9-36c1f24187e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:03.716781 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2444ecf-961d-425e-9ef9-36c1f24187e0 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:03.718367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:03.729267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 997/1957] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:45:03.733813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2444ecf-961d-425e-9ef9-36c1f24187e0 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:03.744241 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2444ecf-961d-425e-9ef9-36c1f24187e0 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:03.749292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302bc5a-715e-48f2-8359-76b85307624c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:03.749922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302bc5a-715e-48f2-8359-76b85307624c admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:03.750202 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302bc5a-715e-48f2-8359-76b85307624c admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:03.750474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302bc5a-715e-48f2-8359-76b85307624c admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:03.751100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302bc5a-715e-48f2-8359-76b85307624c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:03.753061 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302bc5a-715e-48f2-8359-76b85307624c admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:03.874093 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 998/1958] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:45:03 2022] POST /identity/v3/projects => generated 379 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:45:03.878356 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302bc5a-715e-48f2-8359-76b85307624c admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:03.907742 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5704c69c-2045-4254-a910-453f1a45b78f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb6d101335a046498de9cda35ab3bc03'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:03.916761 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5704c69c-2045-4254-a910-453f1a45b78f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb6d101335a046498de9cda35ab3bc03'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:03.918433 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5704c69c-2045-4254-a910-453f1a45b78f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb6d101335a046498de9cda35ab3bc03'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:03.937429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5704c69c-2045-4254-a910-453f1a45b78f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb6d101335a046498de9cda35ab3bc03'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:03.938224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5704c69c-2045-4254-a910-453f1a45b78f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb6d101335a046498de9cda35ab3bc03'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:03.940214 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:03.947401 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302bc5a-715e-48f2-8359-76b85307624c admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:03.952190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813062e-06ce-470d-8208-139d15d98337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:03.952836 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813062e-06ce-470d-8208-139d15d98337 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:03.953120 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813062e-06ce-470d-8208-139d15d98337 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:03.953390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813062e-06ce-470d-8208-139d15d98337 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:03.954092 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813062e-06ce-470d-8208-139d15d98337 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:03.956190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813062e-06ce-470d-8208-139d15d98337 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:04.073451 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 961/1959] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:45:03 2022] DELETE /identity/v3/users/fb6d101335a046498de9cda35ab3bc03 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:04.073844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 999/1960] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:45:03 2022] POST /identity/v3/users => generated 290 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:45:04.084560 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704c69c-2045-4254-a910-453f1a45b78f admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:04.086036 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-5704c69c-2045-4254-a910-453f1a45b78f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:04.156013 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704c69c-2045-4254-a910-453f1a45b78f admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:04.161496 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8701b-5a21-4170-a218-e981525056e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:04.162147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8701b-5a21-4170-a218-e981525056e5 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:04.162425 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8701b-5a21-4170-a218-e981525056e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:04.162717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8701b-5a21-4170-a218-e981525056e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:04.163348 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8701b-5a21-4170-a218-e981525056e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:04.165047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8701b-5a21-4170-a218-e981525056e5 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:04.175720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 962/1961] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:45:04.180189 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813062e-06ce-470d-8208-139d15d98337 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:04.190599 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813062e-06ce-470d-8208-139d15d98337 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:04.195607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2dfcd-a380-4405-b588-713f568283ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:04.196747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2dfcd-a380-4405-b588-713f568283ee admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:04.196747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2dfcd-a380-4405-b588-713f568283ee admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:04.196937 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2dfcd-a380-4405-b588-713f568283ee admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:04.208515 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2dfcd-a380-4405-b588-713f568283ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:04.208515 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2dfcd-a380-4405-b588-713f568283ee admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:04.210723 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1000/1962] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:04.216101 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8701b-5a21-4170-a218-e981525056e5 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:04.226757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8701b-5a21-4170-a218-e981525056e5 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:04.231817 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7910c75-03b0-4b1c-9608-510247c14ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:04.232547 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7910c75-03b0-4b1c-9608-510247c14ef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:04.232831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7910c75-03b0-4b1c-9608-510247c14ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:04.233103 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7910c75-03b0-4b1c-9608-510247c14ef1 admin admin] PATH_INFO: `/v3/projects/3484eb9c91264de097a2e04aec820020/users/b63113425c4e4f63ac040159762e9caf/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:04.250417 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7910c75-03b0-4b1c-9608-510247c14ef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3484eb9c91264de097a2e04aec820020, user_id=b63113425c4e4f63ac040159762e9caf, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:04.253148 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7910c75-03b0-4b1c-9608-510247c14ef1 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:04.407807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2dfcd-a380-4405-b588-713f568283ee admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:04.409715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-44e2dfcd-a380-4405-b588-713f568283ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:04.411106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 963/1963] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:04 2022] PUT /identity/v3/projects/3484eb9c91264de097a2e04aec820020/users/b63113425c4e4f63ac040159762e9caf/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:04.429996 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2dfcd-a380-4405-b588-713f568283ee admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:04.430897 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a5367-6fd6-4228-9834-67b6fa0ee66a None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:04.433698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a5367-6fd6-4228-9834-67b6fa0ee66a None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:04.434135 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a5367-6fd6-4228-9834-67b6fa0ee66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:04.479824 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-192a5367-6fd6-4228-9834-67b6fa0ee66a None None] MFA Rules not processed for user `b63113425c4e4f63ac040159762e9caf`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:04.481225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-192a5367-6fd6-4228-9834-67b6fa0ee66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:04.507308 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec408eb3-5c78-4840-af95-205d25294219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:04.508261 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec408eb3-5c78-4840-af95-205d25294219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:04.508746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec408eb3-5c78-4840-af95-205d25294219 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:04.509162 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec408eb3-5c78-4840-af95-205d25294219 admin admin] PATH_INFO: `/v3/projects/0d19b10cb4b74e2d9300cc0db76035e0` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:04.511658 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec408eb3-5c78-4840-af95-205d25294219 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d19b10cb4b74e2d9300cc0db76035e0)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:04.513861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec408eb3-5c78-4840-af95-205d25294219 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:04.583062 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 964/1964] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:04.587429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a5367-6fd6-4228-9834-67b6fa0ee66a None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:04.599358 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a5367-6fd6-4228-9834-67b6fa0ee66a None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:04.608258 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a57051-2259-474d-8b5e-117c2db1e61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:04.608891 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a57051-2259-474d-8b5e-117c2db1e61c admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:04.609172 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a57051-2259-474d-8b5e-117c2db1e61c admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:04.609589 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a57051-2259-474d-8b5e-117c2db1e61c admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:04.610229 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a57051-2259-474d-8b5e-117c2db1e61c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:04.612362 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a57051-2259-474d-8b5e-117c2db1e61c admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:04.670822 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ec408eb3-5c78-4840-af95-205d25294219 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d19b10cb4b74e2d9300cc0db76035e0', 'request_id': 'req-ec408eb3-5c78-4840-af95-205d25294219'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:04.677990 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ec408eb3-5c78-4840-af95-205d25294219 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d19b10cb4b74e2d9300cc0db76035e0', 'request_id': 'req-ec408eb3-5c78-4840-af95-205d25294219'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:04.686908 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ec408eb3-5c78-4840-af95-205d25294219 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d19b10cb4b74e2d9300cc0db76035e0', 'request_id': 'req-ec408eb3-5c78-4840-af95-205d25294219'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:04.755911 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 965/1965] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:45:04 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:45:04.758742 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a57051-2259-474d-8b5e-117c2db1e61c admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:04.777409 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:04.785462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ec408eb3-5c78-4840-af95-205d25294219 admin admin] The token cache is being invalidate because project 0d19b10cb4b74e2d9300cc0db76035e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:45:04.785745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ec408eb3-5c78-4840-af95-205d25294219 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:04.786453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ec408eb3-5c78-4840-af95-205d25294219 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:04.790161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1001/1966] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:45:04 2022] DELETE /identity/v3/projects/0d19b10cb4b74e2d9300cc0db76035e0 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:04.795372 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b078e8-8f9e-4c01-a94e-932b7b37baf8 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:04.795767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b078e8-8f9e-4c01-a94e-932b7b37baf8 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:04.796127 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b078e8-8f9e-4c01-a94e-932b7b37baf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:04.822635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-70b078e8-8f9e-4c01-a94e-932b7b37baf8 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:04.824884 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-70b078e8-8f9e-4c01-a94e-932b7b37baf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:04.841486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a57051-2259-474d-8b5e-117c2db1e61c admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:04.847486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38767c-f69f-4725-8b44-aa23c1e41657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:04.848464 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38767c-f69f-4725-8b44-aa23c1e41657 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:04.849067 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38767c-f69f-4725-8b44-aa23c1e41657 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:04.849503 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38767c-f69f-4725-8b44-aa23c1e41657 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:04.851298 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:04.851829 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38767c-f69f-4725-8b44-aa23c1e41657 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:04.854881 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38767c-f69f-4725-8b44-aa23c1e41657 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:04.893236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1002/1967] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:04.893987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b078e8-8f9e-4c01-a94e-932b7b37baf8 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:04.895638 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-70b078e8-8f9e-4c01-a94e-932b7b37baf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:04.956068 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b078e8-8f9e-4c01-a94e-932b7b37baf8 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:04.960217 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66486af-4f38-4121-9b56-64bddcfe056d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:04.960823 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66486af-4f38-4121-9b56-64bddcfe056d service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:04.961103 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66486af-4f38-4121-9b56-64bddcfe056d service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:04.961375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66486af-4f38-4121-9b56-64bddcfe056d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:04.963836 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c66486af-4f38-4121-9b56-64bddcfe056d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:04.978155 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 966/1968] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:45:04 2022] POST /identity/v3/users => generated 296 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:04.978155 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39f84b-d954-4837-bebe-158d4b623772 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:04.978155 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39f84b-d954-4837-bebe-158d4b623772 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:04.978155 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39f84b-d954-4837-bebe-158d4b623772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:04.997415 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-ff39f84b-d954-4837-bebe-158d4b623772 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:04.998582 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ff39f84b-d954-4837-bebe-158d4b623772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:05.009199 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 967/1969] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:05.010649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39f84b-d954-4837-bebe-158d4b623772 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:05.016873 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ff39f84b-d954-4837-bebe-158d4b623772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:05.024864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39f84b-d954-4837-bebe-158d4b623772 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:05.031355 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19c24f-0b34-4dca-9767-6a3141e7248f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:05.031961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19c24f-0b34-4dca-9767-6a3141e7248f admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:05.032277 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19c24f-0b34-4dca-9767-6a3141e7248f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:05.032563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19c24f-0b34-4dca-9767-6a3141e7248f admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:05.033212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19c24f-0b34-4dca-9767-6a3141e7248f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:05.034482 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66486af-4f38-4121-9b56-64bddcfe056d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:05.034996 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19c24f-0b34-4dca-9767-6a3141e7248f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:05.035985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66486af-4f38-4121-9b56-64bddcfe056d service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:05.045734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 968/1970] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:05.045734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19c24f-0b34-4dca-9767-6a3141e7248f admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:05.057213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1003/1971] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:05.057213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66486af-4f38-4121-9b56-64bddcfe056d service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:05.057978 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19c24f-0b34-4dca-9767-6a3141e7248f admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:05.061231 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66486af-4f38-4121-9b56-64bddcfe056d service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:05.065151 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70cd94-2553-44e7-ab7c-0edad48a5359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:05.065828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70cd94-2553-44e7-ab7c-0edad48a5359 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:05.066167 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70cd94-2553-44e7-ab7c-0edad48a5359 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:05.066429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70cd94-2553-44e7-ab7c-0edad48a5359 admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:05.066935 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19f512-23d2-424d-b235-03e011c75ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:05.067088 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b70cd94-2553-44e7-ab7c-0edad48a5359 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:45:05.067447 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b70cd94-2553-44e7-ab7c-0edad48a5359 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:05.067874 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19f512-23d2-424d-b235-03e011c75ffc admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:05.068293 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19f512-23d2-424d-b235-03e011c75ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:05.068665 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19f512-23d2-424d-b235-03e011c75ffc admin admin] PATH_INFO: `/v3/projects/83f815906686401ab0b3776ddd15e0b5/users/2b1a26ced6654efdb77e88e334a60fa1/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:05.069150 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b70cd94-2553-44e7-ab7c-0edad48a5359 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:05.084624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 969/1972] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:45:05.089221 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19f512-23d2-424d-b235-03e011c75ffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=83f815906686401ab0b3776ddd15e0b5, user_id=2b1a26ced6654efdb77e88e334a60fa1, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:05.089703 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70cd94-2553-44e7-ab7c-0edad48a5359 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:05.092045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19f512-23d2-424d-b235-03e011c75ffc admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:05.099748 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70cd94-2553-44e7-ab7c-0edad48a5359 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:05.104676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e786c2-e48d-4d09-baa8-6e0ece1125e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:05.105345 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e786c2-e48d-4d09-baa8-6e0ece1125e3 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:05.112301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e786c2-e48d-4d09-baa8-6e0ece1125e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:05.112301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e786c2-e48d-4d09-baa8-6e0ece1125e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:05.112301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e786c2-e48d-4d09-baa8-6e0ece1125e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:05.112301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e786c2-e48d-4d09-baa8-6e0ece1125e3 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:05.226935 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1004/1973] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:05 2022] PUT /identity/v3/projects/83f815906686401ab0b3776ddd15e0b5/users/2b1a26ced6654efdb77e88e334a60fa1/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:05.231716 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19f512-23d2-424d-b235-03e011c75ffc admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:05.232072 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 970/1974] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:45:05 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:05.239796 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e786c2-e48d-4d09-baa8-6e0ece1125e3 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:05.297410 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:05.317163 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e786c2-e48d-4d09-baa8-6e0ece1125e3 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:05.317405 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19f512-23d2-424d-b235-03e011c75ffc admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:05.323043 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c44ce9-941c-4e12-9c39-4c7d63a4bb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:05.323255 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4223f42-a2bb-43f7-8b14-776c761b0408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:05.323790 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c44ce9-941c-4e12-9c39-4c7d63a4bb27 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:05.323924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4223f42-a2bb-43f7-8b14-776c761b0408 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:05.324224 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4223f42-a2bb-43f7-8b14-776c761b0408 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:05.324365 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c44ce9-941c-4e12-9c39-4c7d63a4bb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:05.324642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c44ce9-941c-4e12-9c39-4c7d63a4bb27 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:05.324816 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4223f42-a2bb-43f7-8b14-776c761b0408 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:05.325387 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c44ce9-941c-4e12-9c39-4c7d63a4bb27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:05.325577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4223f42-a2bb-43f7-8b14-776c761b0408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:05.342845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c44ce9-941c-4e12-9c39-4c7d63a4bb27 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:05.342983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4223f42-a2bb-43f7-8b14-776c761b0408 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:05.342983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1005/1975] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:45:05.344030 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4223f42-a2bb-43f7-8b14-776c761b0408 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:05.355047 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4223f42-a2bb-43f7-8b14-776c761b0408 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:05.359835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c1725-f220-474f-a943-82b7936c0a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:05.360514 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c1725-f220-474f-a943-82b7936c0a13 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:05.360792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c1725-f220-474f-a943-82b7936c0a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:05.361060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c1725-f220-474f-a943-82b7936c0a13 admin admin] PATH_INFO: `/v3/projects/83f815906686401ab0b3776ddd15e0b5/users/2b1a26ced6654efdb77e88e334a60fa1/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:05.362992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2c1725-f220-474f-a943-82b7936c0a13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83f815906686401ab0b3776ddd15e0b5, user_id=2b1a26ced6654efdb77e88e334a60fa1, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:05.365635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2c1725-f220-474f-a943-82b7936c0a13 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:05.377836 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1006/1976] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:05 2022] PUT /identity/v3/projects/83f815906686401ab0b3776ddd15e0b5/users/2b1a26ced6654efdb77e88e334a60fa1/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:05.388851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332f0e3-0dc8-4d86-9dcb-3132f211069b None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:05.389211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332f0e3-0dc8-4d86-9dcb-3132f211069b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:05.389557 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332f0e3-0dc8-4d86-9dcb-3132f211069b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:05.421340 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-8332f0e3-0dc8-4d86-9dcb-3132f211069b None None] MFA Rules not processed for user `2b1a26ced6654efdb77e88e334a60fa1`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:05.422705 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8332f0e3-0dc8-4d86-9dcb-3132f211069b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:05.462652 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 971/1977] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:45:05 2022] POST /identity/v3/users => generated 292 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:45:05.470208 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c44ce9-941c-4e12-9c39-4c7d63a4bb27 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:05.481347 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c44ce9-941c-4e12-9c39-4c7d63a4bb27 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:05.503655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1007/1978] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:05 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:45:05.516063 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332f0e3-0dc8-4d86-9dcb-3132f211069b None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:05.520177 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8332f0e3-0dc8-4d86-9dcb-3132f211069b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:05.540509 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:05.548406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8b628-a058-4a6d-a0d8-e9e78efba7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:05.549023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8b628-a058-4a6d-a0d8-e9e78efba7db admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:05.549301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8b628-a058-4a6d-a0d8-e9e78efba7db admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:05.549567 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8b628-a058-4a6d-a0d8-e9e78efba7db admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:05.550239 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8b628-a058-4a6d-a0d8-e9e78efba7db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:05.551928 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8b628-a058-4a6d-a0d8-e9e78efba7db admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:05.565049 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 972/1979] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:05.569031 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8b628-a058-4a6d-a0d8-e9e78efba7db admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:05.579384 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8b628-a058-4a6d-a0d8-e9e78efba7db admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:05.584348 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db717a-df9f-4303-90fc-108fae4cd570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:05.584972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db717a-df9f-4303-90fc-108fae4cd570 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:05.585248 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db717a-df9f-4303-90fc-108fae4cd570 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:05.585519 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db717a-df9f-4303-90fc-108fae4cd570 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:05.586151 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db717a-df9f-4303-90fc-108fae4cd570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:05.587926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db717a-df9f-4303-90fc-108fae4cd570 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:05.595491 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332f0e3-0dc8-4d86-9dcb-3132f211069b None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:05.599313 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5265d6-14be-4196-a209-214b3ef923a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:05.599504 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 973/1980] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:45:05.599927 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5265d6-14be-4196-a209-214b3ef923a2 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:05.600227 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5265d6-14be-4196-a209-214b3ef923a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:05.600494 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5265d6-14be-4196-a209-214b3ef923a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:05.604210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db717a-df9f-4303-90fc-108fae4cd570 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:05.612162 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5265d6-14be-4196-a209-214b3ef923a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:05.613666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5265d6-14be-4196-a209-214b3ef923a2 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:05.614460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db717a-df9f-4303-90fc-108fae4cd570 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:05.620029 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d33c3-27fc-4fe8-8e9f-ca2a65ca312c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:05.620714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d33c3-27fc-4fe8-8e9f-ca2a65ca312c admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:05.620989 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d33c3-27fc-4fe8-8e9f-ca2a65ca312c admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:05.621260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d33c3-27fc-4fe8-8e9f-ca2a65ca312c admin admin] PATH_INFO: `/v3/projects/25379e013e6f4c3394ebfc33e1a85acd/users/526be6dd1dc549cf96a3d44d72d2b4a1/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:05.623835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1008/1981] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:05 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:05.636659 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d33c3-27fc-4fe8-8e9f-ca2a65ca312c admin admin] RBAC: Authorizing `identity:create_grant(project_id=25379e013e6f4c3394ebfc33e1a85acd, user_id=526be6dd1dc549cf96a3d44d72d2b4a1, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:05.639493 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d33c3-27fc-4fe8-8e9f-ca2a65ca312c admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:05.719149 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 974/1982] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:05 2022] PUT /identity/v3/projects/25379e013e6f4c3394ebfc33e1a85acd/users/526be6dd1dc549cf96a3d44d72d2b4a1/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:45:05.731916 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7b7ab-c481-4684-bd9f-6a6bbce1f422 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:05.731916 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7b7ab-c481-4684-bd9f-6a6bbce1f422 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:05.731916 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7b7ab-c481-4684-bd9f-6a6bbce1f422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:05.772491 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-e6a7b7ab-c481-4684-bd9f-6a6bbce1f422 None None] MFA Rules not processed for user `526be6dd1dc549cf96a3d44d72d2b4a1`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:05.773849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e6a7b7ab-c481-4684-bd9f-6a6bbce1f422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:05.883888 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1009/1983] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:05.889069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d33c3-27fc-4fe8-8e9f-ca2a65ca312c admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:05.990180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d33c3-27fc-4fe8-8e9f-ca2a65ca312c admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:05.994942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7beeba2-6fd7-43a1-a6a2-fae5b08c504a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:05.995573 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7beeba2-6fd7-43a1-a6a2-fae5b08c504a admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:05.995852 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7beeba2-6fd7-43a1-a6a2-fae5b08c504a admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:05.996117 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7beeba2-6fd7-43a1-a6a2-fae5b08c504a admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:05.996769 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7beeba2-6fd7-43a1-a6a2-fae5b08c504a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:05.998642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7beeba2-6fd7-43a1-a6a2-fae5b08c504a admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:06.134067 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 975/1984] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:45:05 2022] POST /identity/v3/projects => generated 383 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:06.138736 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7b7ab-c481-4684-bd9f-6a6bbce1f422 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:06.149936 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7b7ab-c481-4684-bd9f-6a6bbce1f422 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:06.216970 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c1a4a-0570-403e-8b3c-cf86c7c67f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:06.217710 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c1a4a-0570-403e-8b3c-cf86c7c67f4e admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:06.221142 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c1a4a-0570-403e-8b3c-cf86c7c67f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:06.221142 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c1a4a-0570-403e-8b3c-cf86c7c67f4e admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:06.221142 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c1a4a-0570-403e-8b3c-cf86c7c67f4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:06.221142 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c1a4a-0570-403e-8b3c-cf86c7c67f4e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:06.244851 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7beeba2-6fd7-43a1-a6a2-fae5b08c504a admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:06.256675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7beeba2-6fd7-43a1-a6a2-fae5b08c504a admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:06.262330 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f62b2b-3a24-4167-86f7-97cca1220e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:06.262965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f62b2b-3a24-4167-86f7-97cca1220e62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:06.263241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f62b2b-3a24-4167-86f7-97cca1220e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:06.263508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f62b2b-3a24-4167-86f7-97cca1220e62 admin admin] PATH_INFO: `/v3/users/b63113425c4e4f63ac040159762e9caf` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:06.264702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f62b2b-3a24-4167-86f7-97cca1220e62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b63113425c4e4f63ac040159762e9caf)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:06.266689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f62b2b-3a24-4167-86f7-97cca1220e62 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:06.314845 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1010/1985] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:45:06 2022] POST /identity/v3/users => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:06.319544 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c1a4a-0570-403e-8b3c-cf86c7c67f4e admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:06.330458 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c1a4a-0570-403e-8b3c-cf86c7c67f4e admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:06.335600 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cc600-d0a7-4bd2-a4db-9aa2db226db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:06.346751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cc600-d0a7-4bd2-a4db-9aa2db226db4 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:06.346751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cc600-d0a7-4bd2-a4db-9aa2db226db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:06.346751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cc600-d0a7-4bd2-a4db-9aa2db226db4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:06.346751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cc600-d0a7-4bd2-a4db-9aa2db226db4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:06.346751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cc600-d0a7-4bd2-a4db-9aa2db226db4 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:06.349458 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1011/1986] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:06.354091 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cc600-d0a7-4bd2-a4db-9aa2db226db4 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:06.365474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cc600-d0a7-4bd2-a4db-9aa2db226db4 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:06.376223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7765d-1336-4d76-a087-359da9ab0a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:06.377079 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7765d-1336-4d76-a087-359da9ab0a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:06.378497 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7765d-1336-4d76-a087-359da9ab0a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:06.378926 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7765d-1336-4d76-a087-359da9ab0a2d admin admin] PATH_INFO: `/v3/projects/c1065360201b44a38742d2d410d5101e/users/82b7069d483a4215b15bd36972708acd/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:06.383316 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:06.401275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7765d-1336-4d76-a087-359da9ab0a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1065360201b44a38742d2d410d5101e, user_id=82b7069d483a4215b15bd36972708acd, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:06.405259 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7765d-1336-4d76-a087-359da9ab0a2d admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:06.506417 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-30f62b2b-3a24-4167-86f7-97cca1220e62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b63113425c4e4f63ac040159762e9caf'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:06.514900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-30f62b2b-3a24-4167-86f7-97cca1220e62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b63113425c4e4f63ac040159762e9caf'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:06.515849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-30f62b2b-3a24-4167-86f7-97cca1220e62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b63113425c4e4f63ac040159762e9caf'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:06.524438 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1012/1987] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:06 2022] PUT /identity/v3/projects/c1065360201b44a38742d2d410d5101e/users/82b7069d483a4215b15bd36972708acd/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:45:06.528684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7765d-1336-4d76-a087-359da9ab0a2d admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:06.530412 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-30f62b2b-3a24-4167-86f7-97cca1220e62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b63113425c4e4f63ac040159762e9caf'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:06.530558 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-95c7765d-1336-4d76-a087-359da9ab0a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:06.531115 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-30f62b2b-3a24-4167-86f7-97cca1220e62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b63113425c4e4f63ac040159762e9caf'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:06.598002 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7765d-1336-4d76-a087-359da9ab0a2d admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:06.603932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5ecaf-a606-48a6-86fa-e01602ee3e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:06.604819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5ecaf-a606-48a6-86fa-e01602ee3e99 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:06.605109 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5ecaf-a606-48a6-86fa-e01602ee3e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:06.605431 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5ecaf-a606-48a6-86fa-e01602ee3e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:06.606123 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb5ecaf-a606-48a6-86fa-e01602ee3e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:06.607787 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb5ecaf-a606-48a6-86fa-e01602ee3e99 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:06.618326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1013/1988] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:45:06.623333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5ecaf-a606-48a6-86fa-e01602ee3e99 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:06.633787 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5ecaf-a606-48a6-86fa-e01602ee3e99 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:06.638826 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc2605-8acc-4c46-9c16-b5065c07b3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:06.639473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc2605-8acc-4c46-9c16-b5065c07b3fe admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:06.639754 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc2605-8acc-4c46-9c16-b5065c07b3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:06.640030 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc2605-8acc-4c46-9c16-b5065c07b3fe admin admin] PATH_INFO: `/v3/projects/c1065360201b44a38742d2d410d5101e/users/82b7069d483a4215b15bd36972708acd/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:06.641984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc2605-8acc-4c46-9c16-b5065c07b3fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1065360201b44a38742d2d410d5101e, user_id=82b7069d483a4215b15bd36972708acd, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:06.644683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc2605-8acc-4c46-9c16-b5065c07b3fe admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:06.655587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1014/1989] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:06 2022] PUT /identity/v3/projects/c1065360201b44a38742d2d410d5101e/users/82b7069d483a4215b15bd36972708acd/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:45:06.667788 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db3eb9-bafe-4f23-9e1a-3f9c97021234 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:06.668093 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db3eb9-bafe-4f23-9e1a-3f9c97021234 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:06.668399 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db3eb9-bafe-4f23-9e1a-3f9c97021234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:06.679351 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 976/1990] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:45:06 2022] DELETE /identity/v3/users/b63113425c4e4f63ac040159762e9caf => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:06.682925 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd496f1-db5f-4ade-82f7-cb219dff6c19 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:06.683400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd496f1-db5f-4ade-82f7-cb219dff6c19 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:06.683737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd496f1-db5f-4ade-82f7-cb219dff6c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:06.704638 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-edd496f1-db5f-4ade-82f7-cb219dff6c19 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:06.705874 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-edd496f1-db5f-4ade-82f7-cb219dff6c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:06.708168 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-c1db3eb9-bafe-4f23-9e1a-3f9c97021234 None None] MFA Rules not processed for user `82b7069d483a4215b15bd36972708acd`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:06.709337 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c1db3eb9-bafe-4f23-9e1a-3f9c97021234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:06.718253 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:06.795450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1015/1991] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:06.800434 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 977/1992] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:06.810419 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db3eb9-bafe-4f23-9e1a-3f9c97021234 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:06.810786 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd496f1-db5f-4ade-82f7-cb219dff6c19 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:06.812175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-edd496f1-db5f-4ade-82f7-cb219dff6c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:06.812292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c1db3eb9-bafe-4f23-9e1a-3f9c97021234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:06.874316 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:06.878200 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db3eb9-bafe-4f23-9e1a-3f9c97021234 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:06.881161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd496f1-db5f-4ade-82f7-cb219dff6c19 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:06.885003 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e0476-134f-4ad0-85f4-ed91302a3cf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:06.885003 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e0476-134f-4ad0-85f4-ed91302a3cf4 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:06.885003 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e0476-134f-4ad0-85f4-ed91302a3cf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:06.885003 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e0476-134f-4ad0-85f4-ed91302a3cf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:06.885346 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-752e0476-134f-4ad0-85f4-ed91302a3cf4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:06.885446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f3588-d2e9-4289-baa9-92fd1d4a8eb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:06.885709 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f3588-d2e9-4289-baa9-92fd1d4a8eb3 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:06.885992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f3588-d2e9-4289-baa9-92fd1d4a8eb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:06.886262 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f3588-d2e9-4289-baa9-92fd1d4a8eb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:06.896734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f3588-d2e9-4289-baa9-92fd1d4a8eb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:06.898337 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f3588-d2e9-4289-baa9-92fd1d4a8eb3 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:06.910606 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 978/1993] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:06.967474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e0476-134f-4ad0-85f4-ed91302a3cf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:06.969257 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e0476-134f-4ad0-85f4-ed91302a3cf4 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:06.979862 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1016/1994] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:07.420011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f3588-d2e9-4289-baa9-92fd1d4a8eb3 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:07.427709 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-0a7f3588-d2e9-4289-baa9-92fd1d4a8eb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:07.447707 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f3588-d2e9-4289-baa9-92fd1d4a8eb3 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:07.452941 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960abf19-e598-491f-ba23-f0a370b58874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:07.453767 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960abf19-e598-491f-ba23-f0a370b58874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:07.454056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960abf19-e598-491f-ba23-f0a370b58874 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:07.455402 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960abf19-e598-491f-ba23-f0a370b58874 admin admin] PATH_INFO: `/v3/projects/3484eb9c91264de097a2e04aec820020` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:07.455667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960abf19-e598-491f-ba23-f0a370b58874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3484eb9c91264de097a2e04aec820020)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:07.458884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960abf19-e598-491f-ba23-f0a370b58874 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:07.576464 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-960abf19-e598-491f-ba23-f0a370b58874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3484eb9c91264de097a2e04aec820020', 'request_id': 'req-960abf19-e598-491f-ba23-f0a370b58874'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:07.645216 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-960abf19-e598-491f-ba23-f0a370b58874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3484eb9c91264de097a2e04aec820020', 'request_id': 'req-960abf19-e598-491f-ba23-f0a370b58874'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:07.653436 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-960abf19-e598-491f-ba23-f0a370b58874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3484eb9c91264de097a2e04aec820020', 'request_id': 'req-960abf19-e598-491f-ba23-f0a370b58874'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:07.665558 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-960abf19-e598-491f-ba23-f0a370b58874 admin admin] The token cache is being invalidate because project 3484eb9c91264de097a2e04aec820020 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:45:07.666708 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-960abf19-e598-491f-ba23-f0a370b58874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:07.667456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-960abf19-e598-491f-ba23-f0a370b58874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:07.669122 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 979/1995] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:45:07 2022] DELETE /identity/v3/projects/3484eb9c91264de097a2e04aec820020 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:07.673318 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e0476-134f-4ad0-85f4-ed91302a3cf4 service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:07.684619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e0476-134f-4ad0-85f4-ed91302a3cf4 service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:07.704913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:07.747508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850830d-9d91-443e-b142-17491f93fe38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:07.748455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850830d-9d91-443e-b142-17491f93fe38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:07.748849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850830d-9d91-443e-b142-17491f93fe38 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:07.749242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850830d-9d91-443e-b142-17491f93fe38 admin admin] PATH_INFO: `/v3/users/2b1a26ced6654efdb77e88e334a60fa1` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:07.750698 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850830d-9d91-443e-b142-17491f93fe38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b1a26ced6654efdb77e88e334a60fa1)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:07.753567 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850830d-9d91-443e-b142-17491f93fe38 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:08.025945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0850830d-9d91-443e-b142-17491f93fe38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b1a26ced6654efdb77e88e334a60fa1'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:08.026664 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0850830d-9d91-443e-b142-17491f93fe38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b1a26ced6654efdb77e88e334a60fa1'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:08.033913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0850830d-9d91-443e-b142-17491f93fe38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b1a26ced6654efdb77e88e334a60fa1'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:08.112093 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0850830d-9d91-443e-b142-17491f93fe38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b1a26ced6654efdb77e88e334a60fa1'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:08.112756 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0850830d-9d91-443e-b142-17491f93fe38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b1a26ced6654efdb77e88e334a60fa1'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:08.132092 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1017/1996] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:45:07 2022] DELETE /identity/v3/users/2b1a26ced6654efdb77e88e334a60fa1 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:45:08.671737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960abf19-e598-491f-ba23-f0a370b58874 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:08.673917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-960abf19-e598-491f-ba23-f0a370b58874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:08.690746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960abf19-e598-491f-ba23-f0a370b58874 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:08.738737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:08.755661 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0affbe4-38e3-471f-a3b9-0ecbe4a2e7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:08.756393 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0affbe4-38e3-471f-a3b9-0ecbe4a2e7d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:08.756672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0affbe4-38e3-471f-a3b9-0ecbe4a2e7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:08.756969 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0affbe4-38e3-471f-a3b9-0ecbe4a2e7d9 admin admin] PATH_INFO: `/v3/projects/83f815906686401ab0b3776ddd15e0b5` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:08.758091 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0affbe4-38e3-471f-a3b9-0ecbe4a2e7d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83f815906686401ab0b3776ddd15e0b5)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:08.760667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0affbe4-38e3-471f-a3b9-0ecbe4a2e7d9 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:08.943273 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b0affbe4-38e3-471f-a3b9-0ecbe4a2e7d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83f815906686401ab0b3776ddd15e0b5', 'request_id': 'req-b0affbe4-38e3-471f-a3b9-0ecbe4a2e7d9'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:09.026370 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b0affbe4-38e3-471f-a3b9-0ecbe4a2e7d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83f815906686401ab0b3776ddd15e0b5', 'request_id': 'req-b0affbe4-38e3-471f-a3b9-0ecbe4a2e7d9'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:09.033338 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b0affbe4-38e3-471f-a3b9-0ecbe4a2e7d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83f815906686401ab0b3776ddd15e0b5', 'request_id': 'req-b0affbe4-38e3-471f-a3b9-0ecbe4a2e7d9'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:09.047762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b0affbe4-38e3-471f-a3b9-0ecbe4a2e7d9 admin admin] The token cache is being invalidate because project 83f815906686401ab0b3776ddd15e0b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:45:09.048184 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b0affbe4-38e3-471f-a3b9-0ecbe4a2e7d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:09.048890 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b0affbe4-38e3-471f-a3b9-0ecbe4a2e7d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:09.051091 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 980/1997] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:45:08 2022] DELETE /identity/v3/projects/83f815906686401ab0b3776ddd15e0b5 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:09.076732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dedd4e-9cc1-48d3-806c-0d3af1b6f656 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:09.076732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dedd4e-9cc1-48d3-806c-0d3af1b6f656 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:09.076732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dedd4e-9cc1-48d3-806c-0d3af1b6f656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:09.078405 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-c1dedd4e-9cc1-48d3-806c-0d3af1b6f656 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:09.079614 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c1dedd4e-9cc1-48d3-806c-0d3af1b6f656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:09.142279 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1018/1998] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:09.150681 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8b8f3-1a52-4f39-b144-02e6e9bc9a06 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:09.150988 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8b8f3-1a52-4f39-b144-02e6e9bc9a06 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:09.151246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8b8f3-1a52-4f39-b144-02e6e9bc9a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:09.172127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-e3f8b8f3-1a52-4f39-b144-02e6e9bc9a06 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:09.173368 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-e3f8b8f3-1a52-4f39-b144-02e6e9bc9a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:09.182084 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 981/1999] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:09.186380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dedd4e-9cc1-48d3-806c-0d3af1b6f656 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:09.198098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dedd4e-9cc1-48d3-806c-0d3af1b6f656 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:09.204531 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528e946-966d-444b-9baa-520f68f1223f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:09.205216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528e946-966d-444b-9baa-520f68f1223f admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:09.205495 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528e946-966d-444b-9baa-520f68f1223f admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:09.205805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528e946-966d-444b-9baa-520f68f1223f admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:09.206486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528e946-966d-444b-9baa-520f68f1223f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:45:09.207052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528e946-966d-444b-9baa-520f68f1223f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:09.208847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528e946-966d-444b-9baa-520f68f1223f admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:09.221955 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1019/2000] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:45:09.227867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8b8f3-1a52-4f39-b144-02e6e9bc9a06 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:09.237435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8b8f3-1a52-4f39-b144-02e6e9bc9a06 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:09.242162 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb08fe4-f20e-4168-89e4-e5887bfd0ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:09.242760 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb08fe4-f20e-4168-89e4-e5887bfd0ea4 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:09.243037 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb08fe4-f20e-4168-89e4-e5887bfd0ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:09.243309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb08fe4-f20e-4168-89e4-e5887bfd0ea4 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:09.244666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb08fe4-f20e-4168-89e4-e5887bfd0ea4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:09.246713 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb08fe4-f20e-4168-89e4-e5887bfd0ea4 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:09.379804 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 982/2001] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:45:09 2022] POST /identity/v3/projects => generated 375 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:09.383779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528e946-966d-444b-9baa-520f68f1223f admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:09.396252 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528e946-966d-444b-9baa-520f68f1223f admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:09.427767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:09.460886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5128f-335f-43b2-bcdd-6f871a194f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:09.461530 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5128f-335f-43b2-bcdd-6f871a194f03 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:09.461806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5128f-335f-43b2-bcdd-6f871a194f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:09.462079 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5128f-335f-43b2-bcdd-6f871a194f03 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:09.462881 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5128f-335f-43b2-bcdd-6f871a194f03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:09.477706 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5128f-335f-43b2-bcdd-6f871a194f03 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:09.578850 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1020/2002] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:45:09 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:09.584019 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb08fe4-f20e-4168-89e4-e5887bfd0ea4 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:09.596228 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb08fe4-f20e-4168-89e4-e5887bfd0ea4 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:09.609607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e401a-52b9-44fd-8c0a-16c7536c9627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:09.611078 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e401a-52b9-44fd-8c0a-16c7536c9627 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:09.611407 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e401a-52b9-44fd-8c0a-16c7536c9627 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:09.611840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e401a-52b9-44fd-8c0a-16c7536c9627 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:09.612954 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e401a-52b9-44fd-8c0a-16c7536c9627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:09.617174 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e401a-52b9-44fd-8c0a-16c7536c9627 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:09.637662 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 983/2003] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:09.643177 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5128f-335f-43b2-bcdd-6f871a194f03 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:09.655493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5128f-335f-43b2-bcdd-6f871a194f03 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:09.660856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48ea0a-de54-4db0-8ca1-2099ea5ad450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:09.662038 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48ea0a-de54-4db0-8ca1-2099ea5ad450 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:09.662381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48ea0a-de54-4db0-8ca1-2099ea5ad450 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:09.665169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48ea0a-de54-4db0-8ca1-2099ea5ad450 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:09.665169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48ea0a-de54-4db0-8ca1-2099ea5ad450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:09.665549 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48ea0a-de54-4db0-8ca1-2099ea5ad450 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:09.684345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1021/2004] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:09.684531 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e401a-52b9-44fd-8c0a-16c7536c9627 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:09.695507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e401a-52b9-44fd-8c0a-16c7536c9627 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:09.700333 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577c9d9-447a-4109-baae-b56a89c3cd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:09.701378 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577c9d9-447a-4109-baae-b56a89c3cd1c admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:09.701510 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577c9d9-447a-4109-baae-b56a89c3cd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:09.702075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577c9d9-447a-4109-baae-b56a89c3cd1c admin admin] PATH_INFO: `/v3/projects/420033d63ac74c6ab0f6ff97d09088f1/users/c777615176614017b08f95a6d5fdc421/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:09.722531 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577c9d9-447a-4109-baae-b56a89c3cd1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=420033d63ac74c6ab0f6ff97d09088f1, user_id=c777615176614017b08f95a6d5fdc421, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:09.726159 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577c9d9-447a-4109-baae-b56a89c3cd1c admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:09.804097 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 984/2005] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:09 2022] PUT /identity/v3/projects/420033d63ac74c6ab0f6ff97d09088f1/users/c777615176614017b08f95a6d5fdc421/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:45:09.814606 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bc7c8-0f2e-4fa8-ad99-a95ae2163000 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:09.814885 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bc7c8-0f2e-4fa8-ad99-a95ae2163000 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:09.815155 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bc7c8-0f2e-4fa8-ad99-a95ae2163000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:09.843741 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-a59bc7c8-0f2e-4fa8-ad99-a95ae2163000 None None] MFA Rules not processed for user `c777615176614017b08f95a6d5fdc421`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:09.845014 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a59bc7c8-0f2e-4fa8-ad99-a95ae2163000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:09.912616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1022/2006] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:09.923975 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577c9d9-447a-4109-baae-b56a89c3cd1c admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:09.925718 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-4577c9d9-447a-4109-baae-b56a89c3cd1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:10.010953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:10.013606 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577c9d9-447a-4109-baae-b56a89c3cd1c admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:10.017420 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41ce19-004a-4a78-bc3f-6d1152f1fce3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4db08aa2e4ef4478a3909a876754814c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:10.021402 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41ce19-004a-4a78-bc3f-6d1152f1fce3 service glance] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:10.021402 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41ce19-004a-4a78-bc3f-6d1152f1fce3 service glance] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:10.021402 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41ce19-004a-4a78-bc3f-6d1152f1fce3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:10.029226 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41ce19-004a-4a78-bc3f-6d1152f1fce3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:10.030673 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41ce19-004a-4a78-bc3f-6d1152f1fce3 service glance] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:10.041519 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 985/2007] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:10.120763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bc7c8-0f2e-4fa8-ad99-a95ae2163000 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:10.123860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a59bc7c8-0f2e-4fa8-ad99-a95ae2163000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:10.209455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bc7c8-0f2e-4fa8-ad99-a95ae2163000 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:10.215510 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486527f-6f43-4725-964b-c6063b5a4769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:10.216246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486527f-6f43-4725-964b-c6063b5a4769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:10.216673 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486527f-6f43-4725-964b-c6063b5a4769 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:10.218122 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486527f-6f43-4725-964b-c6063b5a4769 admin admin] PATH_INFO: `/v3/users/5b560adb99bc4c408d23fe84a8709a9b` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:10.218839 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486527f-6f43-4725-964b-c6063b5a4769 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b560adb99bc4c408d23fe84a8709a9b)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:10.220985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486527f-6f43-4725-964b-c6063b5a4769 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:10.420039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:10.425328 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-9486527f-6f43-4725-964b-c6063b5a4769 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b560adb99bc4c408d23fe84a8709a9b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:10.426003 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-9486527f-6f43-4725-964b-c6063b5a4769 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b560adb99bc4c408d23fe84a8709a9b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:10.434091 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-9486527f-6f43-4725-964b-c6063b5a4769 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b560adb99bc4c408d23fe84a8709a9b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:10.541432 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-9486527f-6f43-4725-964b-c6063b5a4769 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b560adb99bc4c408d23fe84a8709a9b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:10.542088 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-9486527f-6f43-4725-964b-c6063b5a4769 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b560adb99bc4c408d23fe84a8709a9b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:10.564554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1023/2008] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:45:10 2022] DELETE /identity/v3/users/5b560adb99bc4c408d23fe84a8709a9b => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:10.569335 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4d312-4d00-4c68-a1ea-9c08de7c151a None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:10.570234 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4d312-4d00-4c68-a1ea-9c08de7c151a None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:10.570234 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4d312-4d00-4c68-a1ea-9c08de7c151a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:10.592272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-eaa4d312-4d00-4c68-a1ea-9c08de7c151a None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:10.593548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-eaa4d312-4d00-4c68-a1ea-9c08de7c151a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:10.674462 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 986/2009] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:10.693798 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486527f-6f43-4725-964b-c6063b5a4769 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:10.693798 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-9486527f-6f43-4725-964b-c6063b5a4769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:10.752579 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486527f-6f43-4725-964b-c6063b5a4769 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:10.756442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeae9e7-5a14-4d82-b11a-73cef461ea2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:10.757058 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeae9e7-5a14-4d82-b11a-73cef461ea2f service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:10.757330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeae9e7-5a14-4d82-b11a-73cef461ea2f service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:10.757593 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeae9e7-5a14-4d82-b11a-73cef461ea2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:10.768030 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeae9e7-5a14-4d82-b11a-73cef461ea2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:10.769705 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeae9e7-5a14-4d82-b11a-73cef461ea2f service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:10.802939 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1024/2010] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:11.399763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4d312-4d00-4c68-a1ea-9c08de7c151a None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:11.401655 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-eaa4d312-4d00-4c68-a1ea-9c08de7c151a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:11.414921 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4d312-4d00-4c68-a1ea-9c08de7c151a None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:11.420328 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8539c83-e2aa-4e32-a894-7fbf5fbf6e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:11.421370 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8539c83-e2aa-4e32-a894-7fbf5fbf6e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:11.421725 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8539c83-e2aa-4e32-a894-7fbf5fbf6e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:11.422075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8539c83-e2aa-4e32-a894-7fbf5fbf6e25 admin admin] PATH_INFO: `/v3/projects/dd4cbefa02e8484e8f0e9438f96a19f1` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:11.423775 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8539c83-e2aa-4e32-a894-7fbf5fbf6e25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd4cbefa02e8484e8f0e9438f96a19f1)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:11.425513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8539c83-e2aa-4e32-a894-7fbf5fbf6e25 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:11.640323 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d8539c83-e2aa-4e32-a894-7fbf5fbf6e25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd4cbefa02e8484e8f0e9438f96a19f1', 'request_id': 'req-d8539c83-e2aa-4e32-a894-7fbf5fbf6e25'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:11.735291 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d8539c83-e2aa-4e32-a894-7fbf5fbf6e25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd4cbefa02e8484e8f0e9438f96a19f1', 'request_id': 'req-d8539c83-e2aa-4e32-a894-7fbf5fbf6e25'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:11.749475 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d8539c83-e2aa-4e32-a894-7fbf5fbf6e25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd4cbefa02e8484e8f0e9438f96a19f1', 'request_id': 'req-d8539c83-e2aa-4e32-a894-7fbf5fbf6e25'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:11.773353 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d8539c83-e2aa-4e32-a894-7fbf5fbf6e25 admin admin] The token cache is being invalidate because project dd4cbefa02e8484e8f0e9438f96a19f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:45:11.773952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d8539c83-e2aa-4e32-a894-7fbf5fbf6e25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:11.774865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d8539c83-e2aa-4e32-a894-7fbf5fbf6e25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:11.776904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 987/2011] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:45:11 2022] DELETE /identity/v3/projects/dd4cbefa02e8484e8f0e9438f96a19f1 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:11.781090 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeae9e7-5a14-4d82-b11a-73cef461ea2f service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:11.793343 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeae9e7-5a14-4d82-b11a-73cef461ea2f service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:11.875371 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211cd4c-e2f2-4f4e-9568-5688ce9c04cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:11.876059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211cd4c-e2f2-4f4e-9568-5688ce9c04cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:11.876352 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211cd4c-e2f2-4f4e-9568-5688ce9c04cd admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:11.876650 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211cd4c-e2f2-4f4e-9568-5688ce9c04cd admin admin] PATH_INFO: `/v3/users/16a7cd241f50498fa0d334e163d7792a` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:11.877818 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211cd4c-e2f2-4f4e-9568-5688ce9c04cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=16a7cd241f50498fa0d334e163d7792a)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:11.879773 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211cd4c-e2f2-4f4e-9568-5688ce9c04cd admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:12.088572 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-c211cd4c-e2f2-4f4e-9568-5688ce9c04cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16a7cd241f50498fa0d334e163d7792a'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:12.089251 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-c211cd4c-e2f2-4f4e-9568-5688ce9c04cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16a7cd241f50498fa0d334e163d7792a'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:12.097157 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-c211cd4c-e2f2-4f4e-9568-5688ce9c04cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16a7cd241f50498fa0d334e163d7792a'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:12.180574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-c211cd4c-e2f2-4f4e-9568-5688ce9c04cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16a7cd241f50498fa0d334e163d7792a'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:12.181377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-c211cd4c-e2f2-4f4e-9568-5688ce9c04cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16a7cd241f50498fa0d334e163d7792a'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:12.182804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:12.200291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1025/2012] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:45:11 2022] DELETE /identity/v3/users/16a7cd241f50498fa0d334e163d7792a => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:12.656764 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8539c83-e2aa-4e32-a894-7fbf5fbf6e25 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:12.658376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d8539c83-e2aa-4e32-a894-7fbf5fbf6e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:12.679235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8539c83-e2aa-4e32-a894-7fbf5fbf6e25 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:12.758681 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a3446-867f-4743-b465-7c5a08e4edff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:12.759618 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a3446-867f-4743-b465-7c5a08e4edff admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:12.759873 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a3446-867f-4743-b465-7c5a08e4edff admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:12.760185 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a3446-867f-4743-b465-7c5a08e4edff admin admin] PATH_INFO: `/v3/projects/be48a4aa8fcd427f99d5d66c48c722da` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:12.761471 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a3446-867f-4743-b465-7c5a08e4edff admin admin] RBAC: Authorizing `identity:delete_project(project_id=be48a4aa8fcd427f99d5d66c48c722da)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:12.763669 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a3446-867f-4743-b465-7c5a08e4edff admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:12.801632 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:12.901109 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211cd4c-e2f2-4f4e-9568-5688ce9c04cd admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:12.902621 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c211cd4c-e2f2-4f4e-9568-5688ce9c04cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:12.984009 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211cd4c-e2f2-4f4e-9568-5688ce9c04cd admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:13.008259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-807a3446-867f-4743-b465-7c5a08e4edff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be48a4aa8fcd427f99d5d66c48c722da', 'request_id': 'req-807a3446-867f-4743-b465-7c5a08e4edff'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:13.049425 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95787f-2cad-4517-ad81-75eb9cd07dc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:13.050079 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95787f-2cad-4517-ad81-75eb9cd07dc7 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:13.050390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95787f-2cad-4517-ad81-75eb9cd07dc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:13.050666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95787f-2cad-4517-ad81-75eb9cd07dc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:13.052576 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-1c95787f-2cad-4517-ad81-75eb9cd07dc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:13.095180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-807a3446-867f-4743-b465-7c5a08e4edff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be48a4aa8fcd427f99d5d66c48c722da', 'request_id': 'req-807a3446-867f-4743-b465-7c5a08e4edff'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:13.104743 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-807a3446-867f-4743-b465-7c5a08e4edff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be48a4aa8fcd427f99d5d66c48c722da', 'request_id': 'req-807a3446-867f-4743-b465-7c5a08e4edff'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:13.127848 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-807a3446-867f-4743-b465-7c5a08e4edff admin admin] The token cache is being invalidate because project be48a4aa8fcd427f99d5d66c48c722da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:45:13.128709 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-807a3446-867f-4743-b465-7c5a08e4edff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:13.129813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-807a3446-867f-4743-b465-7c5a08e4edff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:13.131407 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 988/2013] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:45:12 2022] DELETE /identity/v3/projects/be48a4aa8fcd427f99d5d66c48c722da => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:13.138104 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a3446-867f-4743-b465-7c5a08e4edff admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:13.139677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-807a3446-867f-4743-b465-7c5a08e4edff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:13.152499 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95787f-2cad-4517-ad81-75eb9cd07dc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:13.153948 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95787f-2cad-4517-ad81-75eb9cd07dc7 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:13.164615 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1026/2014] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:45:13.211465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a3446-867f-4743-b465-7c5a08e4edff admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:13.217385 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c01ca0-dc76-40c0-8158-669ac6bd92fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:13.218117 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c01ca0-dc76-40c0-8158-669ac6bd92fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:13.218476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c01ca0-dc76-40c0-8158-669ac6bd92fe admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:13.218748 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c01ca0-dc76-40c0-8158-669ac6bd92fe admin admin] PATH_INFO: `/v3/users/f5dad0204ef147d18cbeba5b7cdd7f0a` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:13.219935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c01ca0-dc76-40c0-8158-669ac6bd92fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5dad0204ef147d18cbeba5b7cdd7f0a)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:13.222161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c01ca0-dc76-40c0-8158-669ac6bd92fe admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:13.480094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-68c01ca0-dc76-40c0-8158-669ac6bd92fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5dad0204ef147d18cbeba5b7cdd7f0a'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:13.487073 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-68c01ca0-dc76-40c0-8158-669ac6bd92fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5dad0204ef147d18cbeba5b7cdd7f0a'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:13.488076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-68c01ca0-dc76-40c0-8158-669ac6bd92fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5dad0204ef147d18cbeba5b7cdd7f0a'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:13.502842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-68c01ca0-dc76-40c0-8158-669ac6bd92fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5dad0204ef147d18cbeba5b7cdd7f0a'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:13.503575 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-68c01ca0-dc76-40c0-8158-669ac6bd92fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5dad0204ef147d18cbeba5b7cdd7f0a'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:13.610068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 989/2015] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:45:13 2022] DELETE /identity/v3/users/f5dad0204ef147d18cbeba5b7cdd7f0a => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:45:14.120224 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95787f-2cad-4517-ad81-75eb9cd07dc7 service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:14.121783 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-1c95787f-2cad-4517-ad81-75eb9cd07dc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:14.157275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95787f-2cad-4517-ad81-75eb9cd07dc7 service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:14.216034 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:14.231126 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a80ac3-428b-4e64-8f92-5cd2f543ad6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:14.231794 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a80ac3-428b-4e64-8f92-5cd2f543ad6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:14.232074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a80ac3-428b-4e64-8f92-5cd2f543ad6f admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:14.238264 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a80ac3-428b-4e64-8f92-5cd2f543ad6f admin admin] PATH_INFO: `/v3/projects/2c2b0032de4c436b95efa8a3c1af8491` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:14.238264 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a80ac3-428b-4e64-8f92-5cd2f543ad6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c2b0032de4c436b95efa8a3c1af8491)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:14.238264 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a80ac3-428b-4e64-8f92-5cd2f543ad6f admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:14.370641 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-50a80ac3-428b-4e64-8f92-5cd2f543ad6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c2b0032de4c436b95efa8a3c1af8491', 'request_id': 'req-50a80ac3-428b-4e64-8f92-5cd2f543ad6f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:14.377029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-50a80ac3-428b-4e64-8f92-5cd2f543ad6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c2b0032de4c436b95efa8a3c1af8491', 'request_id': 'req-50a80ac3-428b-4e64-8f92-5cd2f543ad6f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:14.382766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-50a80ac3-428b-4e64-8f92-5cd2f543ad6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c2b0032de4c436b95efa8a3c1af8491', 'request_id': 'req-50a80ac3-428b-4e64-8f92-5cd2f543ad6f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:14.460440 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-50a80ac3-428b-4e64-8f92-5cd2f543ad6f admin admin] The token cache is being invalidate because project 2c2b0032de4c436b95efa8a3c1af8491 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:45:14.460760 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-50a80ac3-428b-4e64-8f92-5cd2f543ad6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:14.461625 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-50a80ac3-428b-4e64-8f92-5cd2f543ad6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:14.464535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1027/2016] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:45:14 2022] DELETE /identity/v3/projects/2c2b0032de4c436b95efa8a3c1af8491 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:14.469460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa884a-4b86-43ba-ad67-b6cd1f7e88fa None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:14.469950 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa884a-4b86-43ba-ad67-b6cd1f7e88fa None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:14.470055 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa884a-4b86-43ba-ad67-b6cd1f7e88fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:14.491350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-63fa884a-4b86-43ba-ad67-b6cd1f7e88fa None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:14.492610 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-63fa884a-4b86-43ba-ad67-b6cd1f7e88fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:14.514098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:14.558545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 990/2017] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:14.566538 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa59fb-4fe6-489a-a447-0e236990f1d8 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:14.566839 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa59fb-4fe6-489a-a447-0e236990f1d8 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:14.567109 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa59fb-4fe6-489a-a447-0e236990f1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:14.589754 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-25aa59fb-4fe6-489a-a447-0e236990f1d8 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:14.590957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-25aa59fb-4fe6-489a-a447-0e236990f1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:14.598769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1028/2018] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:14.603218 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa884a-4b86-43ba-ad67-b6cd1f7e88fa None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:14.613222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa884a-4b86-43ba-ad67-b6cd1f7e88fa None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:14.617802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605850e1-8bf4-4c36-ab2d-0df56791fc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:14.618418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605850e1-8bf4-4c36-ab2d-0df56791fc14 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:14.618740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605850e1-8bf4-4c36-ab2d-0df56791fc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:14.619046 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605850e1-8bf4-4c36-ab2d-0df56791fc14 admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:14.619679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605850e1-8bf4-4c36-ab2d-0df56791fc14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:45:14.619972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605850e1-8bf4-4c36-ab2d-0df56791fc14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:14.621594 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605850e1-8bf4-4c36-ab2d-0df56791fc14 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:14.632810 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 991/2019] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:45:14.638006 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa59fb-4fe6-489a-a447-0e236990f1d8 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:14.647931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa59fb-4fe6-489a-a447-0e236990f1d8 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:14.653913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae4693-928d-47cc-bd25-31f3ecad9051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:14.654526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae4693-928d-47cc-bd25-31f3ecad9051 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:14.654800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae4693-928d-47cc-bd25-31f3ecad9051 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:14.655061 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae4693-928d-47cc-bd25-31f3ecad9051 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:14.655680 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae4693-928d-47cc-bd25-31f3ecad9051 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:14.657604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae4693-928d-47cc-bd25-31f3ecad9051 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:14.783146 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1029/2020] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:45:14 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:14.787345 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605850e1-8bf4-4c36-ab2d-0df56791fc14 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:14.797690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605850e1-8bf4-4c36-ab2d-0df56791fc14 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:14.857469 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78523a4e-870f-48ec-bee9-8aeda6e24ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:14.858083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78523a4e-870f-48ec-bee9-8aeda6e24ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:14.858369 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78523a4e-870f-48ec-bee9-8aeda6e24ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:14.858809 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78523a4e-870f-48ec-bee9-8aeda6e24ea8 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:14.859462 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78523a4e-870f-48ec-bee9-8aeda6e24ea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:14.861487 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78523a4e-870f-48ec-bee9-8aeda6e24ea8 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:14.963380 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 992/2021] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:45:14 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:14.967558 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae4693-928d-47cc-bd25-31f3ecad9051 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:14.979838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae4693-928d-47cc-bd25-31f3ecad9051 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:14.985607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5709af-d7c8-4419-938b-d7c8c4a90ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:14.986294 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5709af-d7c8-4419-938b-d7c8c4a90ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:14.986582 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5709af-d7c8-4419-938b-d7c8c4a90ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:14.986912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5709af-d7c8-4419-938b-d7c8c4a90ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:14.987512 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5709af-d7c8-4419-938b-d7c8c4a90ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:14.989249 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5709af-d7c8-4419-938b-d7c8c4a90ff0 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:15.004819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1030/2022] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:15.006188 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78523a4e-870f-48ec-bee9-8aeda6e24ea8 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:15.018665 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78523a4e-870f-48ec-bee9-8aeda6e24ea8 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:15.023707 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd41c6-f217-44e4-a824-e900a5c9a053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:15.024463 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd41c6-f217-44e4-a824-e900a5c9a053 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:15.024773 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd41c6-f217-44e4-a824-e900a5c9a053 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:15.025050 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd41c6-f217-44e4-a824-e900a5c9a053 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:15.025707 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd41c6-f217-44e4-a824-e900a5c9a053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:15.027290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd41c6-f217-44e4-a824-e900a5c9a053 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:15.044178 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 993/2023] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:45:15.046512 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5709af-d7c8-4419-938b-d7c8c4a90ff0 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:15.057711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5709af-d7c8-4419-938b-d7c8c4a90ff0 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:15.062681 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece5e5b-0b92-4a82-a703-2e0e1949215b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:15.063472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece5e5b-0b92-4a82-a703-2e0e1949215b admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:15.064053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece5e5b-0b92-4a82-a703-2e0e1949215b admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:15.064439 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece5e5b-0b92-4a82-a703-2e0e1949215b admin admin] PATH_INFO: `/v3/projects/6a76043a3cd74df8a53c81b1a77568a3/users/45e325a3200b4cd18b15487cc0d28445/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:15.081483 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fece5e5b-0b92-4a82-a703-2e0e1949215b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a76043a3cd74df8a53c81b1a77568a3, user_id=45e325a3200b4cd18b15487cc0d28445, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:15.084876 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fece5e5b-0b92-4a82-a703-2e0e1949215b admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:15.156751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1031/2024] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:15 2022] PUT /identity/v3/projects/6a76043a3cd74df8a53c81b1a77568a3/users/45e325a3200b4cd18b15487cc0d28445/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:45:15.168758 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55a216-c74f-493e-a03a-3e8abe23aded None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:15.169052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55a216-c74f-493e-a03a-3e8abe23aded None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:15.169326 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55a216-c74f-493e-a03a-3e8abe23aded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:15.208615 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-ff55a216-c74f-493e-a03a-3e8abe23aded None None] MFA Rules not processed for user `45e325a3200b4cd18b15487cc0d28445`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:15.210437 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ff55a216-c74f-493e-a03a-3e8abe23aded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:15.294792 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 994/2025] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:15.299407 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece5e5b-0b92-4a82-a703-2e0e1949215b admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:15.389803 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece5e5b-0b92-4a82-a703-2e0e1949215b admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:15.396882 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba450d6-a7f7-4544-893a-688cb53dee8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:15.406366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba450d6-a7f7-4544-893a-688cb53dee8e admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:15.406366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba450d6-a7f7-4544-893a-688cb53dee8e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:15.406366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba450d6-a7f7-4544-893a-688cb53dee8e admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:15.406366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba450d6-a7f7-4544-893a-688cb53dee8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:15.406366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba450d6-a7f7-4544-893a-688cb53dee8e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:15.484200 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:15.499930 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55a216-c74f-493e-a03a-3e8abe23aded None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:15.503121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1032/2026] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:45:15 2022] POST /identity/v3/projects => generated 381 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:15.512005 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55a216-c74f-493e-a03a-3e8abe23aded None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:15.534758 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:15.578988 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290985a-6388-4aca-a0bb-8fd20e774660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:15.583549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290985a-6388-4aca-a0bb-8fd20e774660 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:15.583549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290985a-6388-4aca-a0bb-8fd20e774660 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:15.583549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290985a-6388-4aca-a0bb-8fd20e774660 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:15.583549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290985a-6388-4aca-a0bb-8fd20e774660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:15.585692 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290985a-6388-4aca-a0bb-8fd20e774660 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:15.724356 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 995/2027] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:45:15 2022] POST /identity/v3/users => generated 291 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:15.725914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba450d6-a7f7-4544-893a-688cb53dee8e admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:15.737335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba450d6-a7f7-4544-893a-688cb53dee8e admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:15.742097 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3b32b-d95e-4282-a7fd-1b404d7dca86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:15.742830 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3b32b-d95e-4282-a7fd-1b404d7dca86 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:15.743094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3b32b-d95e-4282-a7fd-1b404d7dca86 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:15.743498 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3b32b-d95e-4282-a7fd-1b404d7dca86 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:15.744296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3b32b-d95e-4282-a7fd-1b404d7dca86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:15.746742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3b32b-d95e-4282-a7fd-1b404d7dca86 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:15.761832 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1033/2028] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:15.766305 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290985a-6388-4aca-a0bb-8fd20e774660 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:15.777457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290985a-6388-4aca-a0bb-8fd20e774660 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:15.782156 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d4108-0f51-44e9-97fc-83cd7a89f8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:15.782762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d4108-0f51-44e9-97fc-83cd7a89f8b0 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:15.783046 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d4108-0f51-44e9-97fc-83cd7a89f8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:15.783329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d4108-0f51-44e9-97fc-83cd7a89f8b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:15.783969 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d4108-0f51-44e9-97fc-83cd7a89f8b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:15.785743 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d4108-0f51-44e9-97fc-83cd7a89f8b0 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:15.797191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 996/2029] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:45:15.803203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3b32b-d95e-4282-a7fd-1b404d7dca86 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:15.812339 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3b32b-d95e-4282-a7fd-1b404d7dca86 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:15.817790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27ab40-37fa-44e8-b970-cb5f639bb0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:15.818444 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27ab40-37fa-44e8-b970-cb5f639bb0e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:15.819107 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27ab40-37fa-44e8-b970-cb5f639bb0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:15.819677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27ab40-37fa-44e8-b970-cb5f639bb0e5 admin admin] PATH_INFO: `/v3/projects/93212f2703334744988911df7988ad9b/users/8bfebb76233d4986959b0acb67f501ff/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:15.836056 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a27ab40-37fa-44e8-b970-cb5f639bb0e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93212f2703334744988911df7988ad9b, user_id=8bfebb76233d4986959b0acb67f501ff, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:15.839011 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a27ab40-37fa-44e8-b970-cb5f639bb0e5 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:15.895261 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1034/2030] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:15 2022] PUT /identity/v3/projects/93212f2703334744988911df7988ad9b/users/8bfebb76233d4986959b0acb67f501ff/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:15.906743 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d5166-053b-4991-b651-4be6e4219e01 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:15.907052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d5166-053b-4991-b651-4be6e4219e01 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:15.907483 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d5166-053b-4991-b651-4be6e4219e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:15.943961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-2f0d5166-053b-4991-b651-4be6e4219e01 None None] MFA Rules not processed for user `8bfebb76233d4986959b0acb67f501ff`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:15.945231 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-2f0d5166-053b-4991-b651-4be6e4219e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:16.019616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 997/2031] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:16.024555 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27ab40-37fa-44e8-b970-cb5f639bb0e5 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:16.103400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27ab40-37fa-44e8-b970-cb5f639bb0e5 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:16.137346 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8149dd-7bf5-45ff-8077-dcc7203d85b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:16.137346 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8149dd-7bf5-45ff-8077-dcc7203d85b3 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:16.137346 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8149dd-7bf5-45ff-8077-dcc7203d85b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:16.137346 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8149dd-7bf5-45ff-8077-dcc7203d85b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:16.137751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8149dd-7bf5-45ff-8077-dcc7203d85b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:16.137751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8149dd-7bf5-45ff-8077-dcc7203d85b3 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:16.216823 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1035/2032] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:45:16 2022] POST /identity/v3/projects => generated 383 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:16.220969 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d5166-053b-4991-b651-4be6e4219e01 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:16.232364 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d5166-053b-4991-b651-4be6e4219e01 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:16.320918 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea802376-9bab-424f-8100-19079e7cae58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:16.321528 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea802376-9bab-424f-8100-19079e7cae58 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:16.321879 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea802376-9bab-424f-8100-19079e7cae58 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:16.322250 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea802376-9bab-424f-8100-19079e7cae58 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:16.323075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea802376-9bab-424f-8100-19079e7cae58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:16.325067 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea802376-9bab-424f-8100-19079e7cae58 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:16.445692 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 998/2033] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:45:16 2022] POST /identity/v3/users => generated 298 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:16.450675 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8149dd-7bf5-45ff-8077-dcc7203d85b3 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:16.461492 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8149dd-7bf5-45ff-8077-dcc7203d85b3 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:16.467016 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac948c3f-0ac0-4a4b-b2c6-a0776fafb428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:16.467645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac948c3f-0ac0-4a4b-b2c6-a0776fafb428 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:16.467926 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac948c3f-0ac0-4a4b-b2c6-a0776fafb428 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:16.468235 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac948c3f-0ac0-4a4b-b2c6-a0776fafb428 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:16.468869 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac948c3f-0ac0-4a4b-b2c6-a0776fafb428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:16.470450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac948c3f-0ac0-4a4b-b2c6-a0776fafb428 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:16.483384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1036/2034] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:16.488574 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea802376-9bab-424f-8100-19079e7cae58 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:16.498636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea802376-9bab-424f-8100-19079e7cae58 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:16.503322 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b257bfc-eba9-470e-8bf9-1ee033326c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:16.503979 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b257bfc-eba9-470e-8bf9-1ee033326c23 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:16.504294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b257bfc-eba9-470e-8bf9-1ee033326c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:16.504567 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b257bfc-eba9-470e-8bf9-1ee033326c23 admin admin] PATH_INFO: `/v3/projects/a0624f041fa4412dbfe294e5fea71867/users/4e04913e6b3c4fe697443c209844f701/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:16.520647 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b257bfc-eba9-470e-8bf9-1ee033326c23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0624f041fa4412dbfe294e5fea71867, user_id=4e04913e6b3c4fe697443c209844f701, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:16.523288 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b257bfc-eba9-470e-8bf9-1ee033326c23 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:16.525746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:16.637969 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 999/2035] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:16 2022] PUT /identity/v3/projects/a0624f041fa4412dbfe294e5fea71867/users/4e04913e6b3c4fe697443c209844f701/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:16.641394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac948c3f-0ac0-4a4b-b2c6-a0776fafb428 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:16.653056 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac948c3f-0ac0-4a4b-b2c6-a0776fafb428 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:16.737976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd144227-997d-4d44-9989-079098691492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:16.738816 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd144227-997d-4d44-9989-079098691492 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:16.739158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd144227-997d-4d44-9989-079098691492 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:16.739519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd144227-997d-4d44-9989-079098691492 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:16.740271 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd144227-997d-4d44-9989-079098691492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:16.741973 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd144227-997d-4d44-9989-079098691492 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:16.759937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b257bfc-eba9-470e-8bf9-1ee033326c23 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:16.761241 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1037/2036] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:16.772764 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b257bfc-eba9-470e-8bf9-1ee033326c23 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:16.778393 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e88fd7-d2b4-4d4a-9a43-73270b3f46b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:16.779061 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e88fd7-d2b4-4d4a-9a43-73270b3f46b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:16.779375 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e88fd7-d2b4-4d4a-9a43-73270b3f46b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:16.779646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e88fd7-d2b4-4d4a-9a43-73270b3f46b9 admin admin] PATH_INFO: `/v3/projects/a0624f041fa4412dbfe294e5fea71867/users/4e04913e6b3c4fe697443c209844f701/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:16.782124 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e88fd7-d2b4-4d4a-9a43-73270b3f46b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0624f041fa4412dbfe294e5fea71867, user_id=4e04913e6b3c4fe697443c209844f701, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:16.784802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e88fd7-d2b4-4d4a-9a43-73270b3f46b9 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:16.797690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1000/2037] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:16 2022] PUT /identity/v3/projects/a0624f041fa4412dbfe294e5fea71867/users/4e04913e6b3c4fe697443c209844f701/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:16.808685 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d29fe1-ea13-4498-87bf-a6c7a55c00b7 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:16.809171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d29fe1-ea13-4498-87bf-a6c7a55c00b7 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:16.809441 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d29fe1-ea13-4498-87bf-a6c7a55c00b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:16.848691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-89d29fe1-ea13-4498-87bf-a6c7a55c00b7 None None] MFA Rules not processed for user `4e04913e6b3c4fe697443c209844f701`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:16.849831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-89d29fe1-ea13-4498-87bf-a6c7a55c00b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:16.872866 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:16.930405 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1038/2038] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:45:16.942157 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e88fd7-d2b4-4d4a-9a43-73270b3f46b9 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:16.943856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-24e88fd7-d2b4-4d4a-9a43-73270b3f46b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:17.015383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e88fd7-d2b4-4d4a-9a43-73270b3f46b9 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:17.023552 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21979d6c-9b53-41ad-8c5a-9d65060178a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:17.024215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21979d6c-9b53-41ad-8c5a-9d65060178a1 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:17.024536 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21979d6c-9b53-41ad-8c5a-9d65060178a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:17.024822 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21979d6c-9b53-41ad-8c5a-9d65060178a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:17.092363 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21979d6c-9b53-41ad-8c5a-9d65060178a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:17.093748 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21979d6c-9b53-41ad-8c5a-9d65060178a1 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:17.109138 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1001/2039] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:19.372308 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d29fe1-ea13-4498-87bf-a6c7a55c00b7 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:19.379135 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-89d29fe1-ea13-4498-87bf-a6c7a55c00b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:19.463627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d29fe1-ea13-4498-87bf-a6c7a55c00b7 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:19.469215 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0928b4-84a1-4ced-ac70-577c2522ed0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:19.469986 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0928b4-84a1-4ced-ac70-577c2522ed0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:19.470345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0928b4-84a1-4ced-ac70-577c2522ed0f admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:19.470608 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0928b4-84a1-4ced-ac70-577c2522ed0f admin admin] PATH_INFO: `/v3/users/526be6dd1dc549cf96a3d44d72d2b4a1` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:19.471791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0928b4-84a1-4ced-ac70-577c2522ed0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=526be6dd1dc549cf96a3d44d72d2b4a1)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:19.473764 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0928b4-84a1-4ced-ac70-577c2522ed0f admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:19.523066 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21979d6c-9b53-41ad-8c5a-9d65060178a1 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:19.528269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-21979d6c-9b53-41ad-8c5a-9d65060178a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:19.547121 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21979d6c-9b53-41ad-8c5a-9d65060178a1 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:19.551270 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a7c1d-aeec-4dff-8d30-c6d170b36450 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:19.551917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a7c1d-aeec-4dff-8d30-c6d170b36450 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:19.552098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a7c1d-aeec-4dff-8d30-c6d170b36450 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:19.552409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a7c1d-aeec-4dff-8d30-c6d170b36450 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:19.640934 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a7c1d-aeec-4dff-8d30-c6d170b36450 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:19.642616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a7c1d-aeec-4dff-8d30-c6d170b36450 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:19.652698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1002/2040] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:19.744071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a7c1d-aeec-4dff-8d30-c6d170b36450 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:19.746430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f92a7c1d-aeec-4dff-8d30-c6d170b36450 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:19.767432 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a7c1d-aeec-4dff-8d30-c6d170b36450 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:19.770615 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ca0928b4-84a1-4ced-ac70-577c2522ed0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '526be6dd1dc549cf96a3d44d72d2b4a1'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:19.771352 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ca0928b4-84a1-4ced-ac70-577c2522ed0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '526be6dd1dc549cf96a3d44d72d2b4a1'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:19.772486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bda4f2-3dc5-4815-a236-f366aa662fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:19.773123 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bda4f2-3dc5-4815-a236-f366aa662fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:19.773393 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bda4f2-3dc5-4815-a236-f366aa662fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:19.773665 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bda4f2-3dc5-4815-a236-f366aa662fc3 admin admin] PATH_INFO: `/v3/users/b21dc8a79bce4b26a690d8990c52116c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:19.775755 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bda4f2-3dc5-4815-a236-f366aa662fc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b21dc8a79bce4b26a690d8990c52116c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:19.777958 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bda4f2-3dc5-4815-a236-f366aa662fc3 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:19.779995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ca0928b4-84a1-4ced-ac70-577c2522ed0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '526be6dd1dc549cf96a3d44d72d2b4a1'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:19.860929 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ca0928b4-84a1-4ced-ac70-577c2522ed0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '526be6dd1dc549cf96a3d44d72d2b4a1'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:19.861655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ca0928b4-84a1-4ced-ac70-577c2522ed0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '526be6dd1dc549cf96a3d44d72d2b4a1'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:19.890730 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:INSERT=2,keystone:SELECT=93,keystone:DELETE=5 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:19.891116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1039/2041] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:45:19 2022] DELETE /identity/v3/users/526be6dd1dc549cf96a3d44d72d2b4a1 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:19.895994 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ce30f-fe23-4f7d-bfae-5f0ae998f020 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:19.896323 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ce30f-fe23-4f7d-bfae-5f0ae998f020 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:19.896596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ce30f-fe23-4f7d-bfae-5f0ae998f020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:19.917802 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-538ce30f-fe23-4f7d-bfae-5f0ae998f020 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:19.919052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-538ce30f-fe23-4f7d-bfae-5f0ae998f020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:19.954654 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:19.981557 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1040/2042] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:45:19.992502 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ce30f-fe23-4f7d-bfae-5f0ae998f020 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:19.994634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-538ce30f-fe23-4f7d-bfae-5f0ae998f020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:20.022677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-46bda4f2-3dc5-4815-a236-f366aa662fc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b21dc8a79bce4b26a690d8990c52116c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:20.033419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-46bda4f2-3dc5-4815-a236-f366aa662fc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b21dc8a79bce4b26a690d8990c52116c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:20.034193 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-46bda4f2-3dc5-4815-a236-f366aa662fc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b21dc8a79bce4b26a690d8990c52116c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:20.049198 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-46bda4f2-3dc5-4815-a236-f366aa662fc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b21dc8a79bce4b26a690d8990c52116c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:20.050976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-46bda4f2-3dc5-4815-a236-f366aa662fc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b21dc8a79bce4b26a690d8990c52116c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:20.078812 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ce30f-fe23-4f7d-bfae-5f0ae998f020 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:20.082717 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a30103-5c45-4e74-9ed1-065a727da7da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:20.083318 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a30103-5c45-4e74-9ed1-065a727da7da service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:20.083595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a30103-5c45-4e74-9ed1-065a727da7da service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:20.083860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a30103-5c45-4e74-9ed1-065a727da7da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:20.085660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-48a30103-5c45-4e74-9ed1-065a727da7da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:20.097996 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a30103-5c45-4e74-9ed1-065a727da7da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:20.099506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a30103-5c45-4e74-9ed1-065a727da7da service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:20.110404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1041/2043] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:20.136807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1003/2044] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:45:19 2022] DELETE /identity/v3/users/b21dc8a79bce4b26a690d8990c52116c => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:20.451611 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a30103-5c45-4e74-9ed1-065a727da7da service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:20.455813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-48a30103-5c45-4e74-9ed1-065a727da7da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:20.475048 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a30103-5c45-4e74-9ed1-065a727da7da service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:20.579720 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc185420-1043-4bbc-bb1f-4aa1b7e673bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:20.580519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc185420-1043-4bbc-bb1f-4aa1b7e673bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:20.580802 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc185420-1043-4bbc-bb1f-4aa1b7e673bd admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:20.581071 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc185420-1043-4bbc-bb1f-4aa1b7e673bd admin admin] PATH_INFO: `/v3/projects/844009bb229a4357bb183c22473ec045` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:20.582624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc185420-1043-4bbc-bb1f-4aa1b7e673bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=844009bb229a4357bb183c22473ec045)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:20.584665 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc185420-1043-4bbc-bb1f-4aa1b7e673bd admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:20.770414 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bda4f2-3dc5-4815-a236-f366aa662fc3 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:20.771953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-46bda4f2-3dc5-4815-a236-f366aa662fc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:20.787993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-fc185420-1043-4bbc-bb1f-4aa1b7e673bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '844009bb229a4357bb183c22473ec045', 'request_id': 'req-fc185420-1043-4bbc-bb1f-4aa1b7e673bd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:20.791868 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bda4f2-3dc5-4815-a236-f366aa662fc3 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:20.795842 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-fc185420-1043-4bbc-bb1f-4aa1b7e673bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '844009bb229a4357bb183c22473ec045', 'request_id': 'req-fc185420-1043-4bbc-bb1f-4aa1b7e673bd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:20.803450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-fc185420-1043-4bbc-bb1f-4aa1b7e673bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '844009bb229a4357bb183c22473ec045', 'request_id': 'req-fc185420-1043-4bbc-bb1f-4aa1b7e673bd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:20.853039 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f6ecb-4324-44d1-9749-bf1aab87eaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:20.853709 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f6ecb-4324-44d1-9749-bf1aab87eaa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:20.853983 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f6ecb-4324-44d1-9749-bf1aab87eaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:20.854269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f6ecb-4324-44d1-9749-bf1aab87eaa2 admin admin] PATH_INFO: `/v3/projects/25379e013e6f4c3394ebfc33e1a85acd` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:20.855439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f6ecb-4324-44d1-9749-bf1aab87eaa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25379e013e6f4c3394ebfc33e1a85acd)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:20.857411 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f6ecb-4324-44d1-9749-bf1aab87eaa2 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:20.893867 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-fc185420-1043-4bbc-bb1f-4aa1b7e673bd admin admin] The token cache is being invalidate because project 844009bb229a4357bb183c22473ec045 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:45:20.894152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-fc185420-1043-4bbc-bb1f-4aa1b7e673bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:20.894783 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-fc185420-1043-4bbc-bb1f-4aa1b7e673bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:20.896679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1042/2045] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:45:20 2022] DELETE /identity/v3/projects/844009bb229a4357bb183c22473ec045 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:20.901848 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fd57e-1767-4acc-b693-8d4f4e0f0705 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:20.902124 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fd57e-1767-4acc-b693-8d4f4e0f0705 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:20.902396 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fd57e-1767-4acc-b693-8d4f4e0f0705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:20.923554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-606fd57e-1767-4acc-b693-8d4f4e0f0705 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:20.924815 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-606fd57e-1767-4acc-b693-8d4f4e0f0705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:20.933616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1043/2046] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:45:20.943225 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb451d15-e5c6-4c5f-b4d0-5a33884e120b None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:20.943504 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb451d15-e5c6-4c5f-b4d0-5a33884e120b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:20.943857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb451d15-e5c6-4c5f-b4d0-5a33884e120b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:20.964590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-eb451d15-e5c6-4c5f-b4d0-5a33884e120b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:20.965694 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-eb451d15-e5c6-4c5f-b4d0-5a33884e120b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:20.975575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1044/2047] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:45:20.984582 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb451d15-e5c6-4c5f-b4d0-5a33884e120b None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:20.997245 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb451d15-e5c6-4c5f-b4d0-5a33884e120b None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:21.002248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca5f42-e841-483f-bb47-20886c692c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:21.002841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca5f42-e841-483f-bb47-20886c692c37 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:21.003115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca5f42-e841-483f-bb47-20886c692c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:21.003382 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca5f42-e841-483f-bb47-20886c692c37 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:21.004017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca5f42-e841-483f-bb47-20886c692c37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:45:21.004349 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca5f42-e841-483f-bb47-20886c692c37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:21.005954 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca5f42-e841-483f-bb47-20886c692c37 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:21.017796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1045/2048] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:45:21.022118 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca5f42-e841-483f-bb47-20886c692c37 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:21.030209 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5a6f6ecb-4324-44d1-9749-bf1aab87eaa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25379e013e6f4c3394ebfc33e1a85acd', 'request_id': 'req-5a6f6ecb-4324-44d1-9749-bf1aab87eaa2'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:21.034344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca5f42-e841-483f-bb47-20886c692c37 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:21.092795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f302f2a-3471-42cd-b4cd-1220c9f74cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:21.093407 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f302f2a-3471-42cd-b4cd-1220c9f74cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:21.093713 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f302f2a-3471-42cd-b4cd-1220c9f74cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:21.094078 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f302f2a-3471-42cd-b4cd-1220c9f74cb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:21.094562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f302f2a-3471-42cd-b4cd-1220c9f74cb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:21.096498 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f302f2a-3471-42cd-b4cd-1220c9f74cb5 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:21.135034 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5a6f6ecb-4324-44d1-9749-bf1aab87eaa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25379e013e6f4c3394ebfc33e1a85acd', 'request_id': 'req-5a6f6ecb-4324-44d1-9749-bf1aab87eaa2'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:21.142277 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5a6f6ecb-4324-44d1-9749-bf1aab87eaa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25379e013e6f4c3394ebfc33e1a85acd', 'request_id': 'req-5a6f6ecb-4324-44d1-9749-bf1aab87eaa2'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:21.154349 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5a6f6ecb-4324-44d1-9749-bf1aab87eaa2 admin admin] The token cache is being invalidate because project 25379e013e6f4c3394ebfc33e1a85acd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:45:21.154634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5a6f6ecb-4324-44d1-9749-bf1aab87eaa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:21.155323 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5a6f6ecb-4324-44d1-9749-bf1aab87eaa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:21.156929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1004/2049] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:45:20 2022] DELETE /identity/v3/projects/25379e013e6f4c3394ebfc33e1a85acd => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:45:21.160943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f6ecb-4324-44d1-9749-bf1aab87eaa2 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:21.162265 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-5a6f6ecb-4324-44d1-9749-bf1aab87eaa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:21.175845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f6ecb-4324-44d1-9749-bf1aab87eaa2 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:21.180710 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a8de7-7e1c-48c7-819b-5042f09149b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:21.181434 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a8de7-7e1c-48c7-819b-5042f09149b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:21.181713 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a8de7-7e1c-48c7-819b-5042f09149b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:21.182011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a8de7-7e1c-48c7-819b-5042f09149b2 admin admin] PATH_INFO: `/v3/users/82b7069d483a4215b15bd36972708acd` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:21.183150 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a8de7-7e1c-48c7-819b-5042f09149b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82b7069d483a4215b15bd36972708acd)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:21.185466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a8de7-7e1c-48c7-819b-5042f09149b2 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:21.199561 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1046/2050] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:45:21 2022] POST /identity/v3/projects => generated 383 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:45:21.204169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f302f2a-3471-42cd-b4cd-1220c9f74cb5 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:21.205951 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6f302f2a-3471-42cd-b4cd-1220c9f74cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:21.277605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f302f2a-3471-42cd-b4cd-1220c9f74cb5 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:21.282978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd51fd3-57e5-4d70-8e1f-c9bb75c9dfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:21.283590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd51fd3-57e5-4d70-8e1f-c9bb75c9dfc3 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:21.283882 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd51fd3-57e5-4d70-8e1f-c9bb75c9dfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:21.284161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd51fd3-57e5-4d70-8e1f-c9bb75c9dfc3 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:21.284795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd51fd3-57e5-4d70-8e1f-c9bb75c9dfc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:21.286824 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd51fd3-57e5-4d70-8e1f-c9bb75c9dfc3 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:21.405690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:21.411854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1047/2051] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:45:21 2022] POST /identity/v3/users => generated 292 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:45:21.417196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd51fd3-57e5-4d70-8e1f-c9bb75c9dfc3 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:21.435118 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd51fd3-57e5-4d70-8e1f-c9bb75c9dfc3 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:21.440916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d62a8de7-7e1c-48c7-819b-5042f09149b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82b7069d483a4215b15bd36972708acd'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:21.445431 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe0e3f-3130-4e0e-8f0b-2548d5337832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:21.445431 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe0e3f-3130-4e0e-8f0b-2548d5337832 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:21.445673 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe0e3f-3130-4e0e-8f0b-2548d5337832 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:21.445865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe0e3f-3130-4e0e-8f0b-2548d5337832 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:21.447498 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fe0e3f-3130-4e0e-8f0b-2548d5337832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:21.449124 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fe0e3f-3130-4e0e-8f0b-2548d5337832 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:21.454879 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d62a8de7-7e1c-48c7-819b-5042f09149b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82b7069d483a4215b15bd36972708acd'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:21.480621 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d62a8de7-7e1c-48c7-819b-5042f09149b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82b7069d483a4215b15bd36972708acd'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:21.480621 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d62a8de7-7e1c-48c7-819b-5042f09149b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82b7069d483a4215b15bd36972708acd'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:21.480621 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-d62a8de7-7e1c-48c7-819b-5042f09149b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82b7069d483a4215b15bd36972708acd'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:21.481539 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1048/2052] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:45:21.482149 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe0e3f-3130-4e0e-8f0b-2548d5337832 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:21.483978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-70fe0e3f-3130-4e0e-8f0b-2548d5337832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:21.497179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe0e3f-3130-4e0e-8f0b-2548d5337832 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:21.500786 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:21.504369 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a5d4b-e303-4c7b-a743-152b0cca9b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:21.504975 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a5d4b-e303-4c7b-a743-152b0cca9b6d admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:21.505247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a5d4b-e303-4c7b-a743-152b0cca9b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:21.505518 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a5d4b-e303-4c7b-a743-152b0cca9b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:21.506192 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923a5d4b-e303-4c7b-a743-152b0cca9b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:21.507792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923a5d4b-e303-4c7b-a743-152b0cca9b6d admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:21.525111 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1049/2053] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:21.526568 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a5d4b-e303-4c7b-a743-152b0cca9b6d admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:21.538778 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a5d4b-e303-4c7b-a743-152b0cca9b6d admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:21.544722 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958424d5-43eb-42a6-be93-646c099b32c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:21.545459 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958424d5-43eb-42a6-be93-646c099b32c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:21.545904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958424d5-43eb-42a6-be93-646c099b32c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:21.546186 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958424d5-43eb-42a6-be93-646c099b32c4 admin admin] PATH_INFO: `/v3/projects/a826c462a3fa429eb866d354afcbe950/users/9c2fdfdb62b54f05b7d81f14ec197cca/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:21.567378 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958424d5-43eb-42a6-be93-646c099b32c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a826c462a3fa429eb866d354afcbe950, user_id=9c2fdfdb62b54f05b7d81f14ec197cca, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:21.570420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958424d5-43eb-42a6-be93-646c099b32c4 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:21.581108 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1005/2054] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:45:21 2022] DELETE /identity/v3/users/82b7069d483a4215b15bd36972708acd => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:45:21.582293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a8de7-7e1c-48c7-819b-5042f09149b2 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:21.584184 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d62a8de7-7e1c-48c7-819b-5042f09149b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:21.672814 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a8de7-7e1c-48c7-819b-5042f09149b2 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:21.677833 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531d50a-d828-484f-8622-98444c8cd702 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:21.678660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531d50a-d828-484f-8622-98444c8cd702 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:21.679004 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531d50a-d828-484f-8622-98444c8cd702 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:21.679334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531d50a-d828-484f-8622-98444c8cd702 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:21.682746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-5531d50a-d828-484f-8622-98444c8cd702 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:21.686192 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1050/2055] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:21 2022] PUT /identity/v3/projects/a826c462a3fa429eb866d354afcbe950/users/9c2fdfdb62b54f05b7d81f14ec197cca/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:21.692056 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e4196-e1ca-4860-97e2-bbdb57ad007c None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:21.692615 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e4196-e1ca-4860-97e2-bbdb57ad007c None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:21.693210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e4196-e1ca-4860-97e2-bbdb57ad007c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:21.725564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-419e4196-e1ca-4860-97e2-bbdb57ad007c None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:21.727794 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-419e4196-e1ca-4860-97e2-bbdb57ad007c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:21.771384 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531d50a-d828-484f-8622-98444c8cd702 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:21.773188 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531d50a-d828-484f-8622-98444c8cd702 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:21.785486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1006/2056] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:21.793491 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1051/2057] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:45:21.803398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531d50a-d828-484f-8622-98444c8cd702 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:21.804981 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-5531d50a-d828-484f-8622-98444c8cd702 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:21.870412 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531d50a-d828-484f-8622-98444c8cd702 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:21.936325 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d8669-6495-4928-b17d-3d6d7a898fcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:21.937354 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d8669-6495-4928-b17d-3d6d7a898fcd service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:21.937641 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d8669-6495-4928-b17d-3d6d7a898fcd service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:21.937911 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d8669-6495-4928-b17d-3d6d7a898fcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:21.948293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d8669-6495-4928-b17d-3d6d7a898fcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:21.949733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d8669-6495-4928-b17d-3d6d7a898fcd service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:21.960360 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1007/2058] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:22.115294 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e4196-e1ca-4860-97e2-bbdb57ad007c None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:22.116723 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-419e4196-e1ca-4860-97e2-bbdb57ad007c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:22.136828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e4196-e1ca-4860-97e2-bbdb57ad007c None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:22.144645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a24f8-30a1-49f4-aba4-c0ba390f7a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:22.145648 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a24f8-30a1-49f4-aba4-c0ba390f7a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:22.146133 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a24f8-30a1-49f4-aba4-c0ba390f7a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:22.146417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a24f8-30a1-49f4-aba4-c0ba390f7a61 admin admin] PATH_INFO: `/v3/projects/c1065360201b44a38742d2d410d5101e` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:22.147796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a24f8-30a1-49f4-aba4-c0ba390f7a61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1065360201b44a38742d2d410d5101e)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:22.150358 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a24f8-30a1-49f4-aba4-c0ba390f7a61 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:22.281748 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-db5a24f8-30a1-49f4-aba4-c0ba390f7a61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1065360201b44a38742d2d410d5101e', 'request_id': 'req-db5a24f8-30a1-49f4-aba4-c0ba390f7a61'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:22.288416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-db5a24f8-30a1-49f4-aba4-c0ba390f7a61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1065360201b44a38742d2d410d5101e', 'request_id': 'req-db5a24f8-30a1-49f4-aba4-c0ba390f7a61'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:22.294868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-db5a24f8-30a1-49f4-aba4-c0ba390f7a61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1065360201b44a38742d2d410d5101e', 'request_id': 'req-db5a24f8-30a1-49f4-aba4-c0ba390f7a61'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:22.408125 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-db5a24f8-30a1-49f4-aba4-c0ba390f7a61 admin admin] The token cache is being invalidate because project c1065360201b44a38742d2d410d5101e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:45:22.408427 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-db5a24f8-30a1-49f4-aba4-c0ba390f7a61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:22.409046 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-db5a24f8-30a1-49f4-aba4-c0ba390f7a61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:22.410600 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1052/2059] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:45:22 2022] DELETE /identity/v3/projects/c1065360201b44a38742d2d410d5101e => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:22.418001 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cd618-6d25-4494-8d52-2be8d29df218 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:22.418351 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cd618-6d25-4494-8d52-2be8d29df218 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:22.418846 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cd618-6d25-4494-8d52-2be8d29df218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:22.438645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-490cd618-6d25-4494-8d52-2be8d29df218 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:22.439865 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-490cd618-6d25-4494-8d52-2be8d29df218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:22.511440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1008/2060] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:22.519432 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec25c7-a174-4b93-a929-194c0b07c34a None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:22.519929 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec25c7-a174-4b93-a929-194c0b07c34a None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:22.520006 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec25c7-a174-4b93-a929-194c0b07c34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:22.544526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-3dec25c7-a174-4b93-a929-194c0b07c34a None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:22.546346 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3dec25c7-a174-4b93-a929-194c0b07c34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:22.561845 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1053/2061] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:45:22.567186 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cd618-6d25-4494-8d52-2be8d29df218 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:22.578619 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cd618-6d25-4494-8d52-2be8d29df218 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:22.591014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5eb7e6-1a09-44fd-885b-1ce44a949f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:22.591767 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5eb7e6-1a09-44fd-885b-1ce44a949f1d admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:22.592129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5eb7e6-1a09-44fd-885b-1ce44a949f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:22.592516 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5eb7e6-1a09-44fd-885b-1ce44a949f1d admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:22.593274 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5eb7e6-1a09-44fd-885b-1ce44a949f1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:45:22.593780 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5eb7e6-1a09-44fd-885b-1ce44a949f1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:22.595559 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5eb7e6-1a09-44fd-885b-1ce44a949f1d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:22.608992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1009/2062] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:45:22.613523 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec25c7-a174-4b93-a929-194c0b07c34a None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:22.626533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec25c7-a174-4b93-a929-194c0b07c34a None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:22.631949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f95d42-daf9-48ec-8623-984fddae3870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:22.632570 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f95d42-daf9-48ec-8623-984fddae3870 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:22.632843 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f95d42-daf9-48ec-8623-984fddae3870 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:22.633134 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f95d42-daf9-48ec-8623-984fddae3870 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:22.633842 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f95d42-daf9-48ec-8623-984fddae3870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:22.636165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f95d42-daf9-48ec-8623-984fddae3870 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:22.868743 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1054/2063] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:45:22 2022] POST /identity/v3/projects => generated 385 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:22.869720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5eb7e6-1a09-44fd-885b-1ce44a949f1d admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:22.881783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5eb7e6-1a09-44fd-885b-1ce44a949f1d admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:22.953909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415333f-6720-4b93-bb06-1f9b906c4b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:22.954529 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415333f-6720-4b93-bb06-1f9b906c4b9a admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:22.954798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415333f-6720-4b93-bb06-1f9b906c4b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:22.955063 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415333f-6720-4b93-bb06-1f9b906c4b9a admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:22.955701 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415333f-6720-4b93-bb06-1f9b906c4b9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:22.957939 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415333f-6720-4b93-bb06-1f9b906c4b9a admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:23.045268 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1010/2064] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:45:22 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:45:23.049727 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f95d42-daf9-48ec-8623-984fddae3870 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:23.062333 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f95d42-daf9-48ec-8623-984fddae3870 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:23.068807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f764169-9584-40ee-9e13-39bc802c7b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:23.070748 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f764169-9584-40ee-9e13-39bc802c7b82 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:23.070748 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f764169-9584-40ee-9e13-39bc802c7b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:23.070748 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f764169-9584-40ee-9e13-39bc802c7b82 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:23.071115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f764169-9584-40ee-9e13-39bc802c7b82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:23.072833 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f764169-9584-40ee-9e13-39bc802c7b82 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:23.087199 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1055/2065] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:23.096464 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415333f-6720-4b93-bb06-1f9b906c4b9a admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:23.108825 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415333f-6720-4b93-bb06-1f9b906c4b9a admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:23.114632 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf41b66-7728-48b2-8f88-55006f04f6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:23.115458 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf41b66-7728-48b2-8f88-55006f04f6b8 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:23.115796 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf41b66-7728-48b2-8f88-55006f04f6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:23.116066 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf41b66-7728-48b2-8f88-55006f04f6b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:23.116748 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf41b66-7728-48b2-8f88-55006f04f6b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:23.119214 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf41b66-7728-48b2-8f88-55006f04f6b8 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:23.136530 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1011/2066] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:23.139043 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f764169-9584-40ee-9e13-39bc802c7b82 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:23.153197 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f764169-9584-40ee-9e13-39bc802c7b82 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:23.176736 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e162c3-2b6a-4c41-bf3d-a43eac9bca26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:23.176736 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e162c3-2b6a-4c41-bf3d-a43eac9bca26 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:23.176736 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e162c3-2b6a-4c41-bf3d-a43eac9bca26 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:23.176736 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e162c3-2b6a-4c41-bf3d-a43eac9bca26 admin admin] PATH_INFO: `/v3/projects/ba1e5cda15ff461ab99b1c49b3ced4f1/users/d312ba6600684240819fd8a3f23ec053/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:23.187898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e162c3-2b6a-4c41-bf3d-a43eac9bca26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba1e5cda15ff461ab99b1c49b3ced4f1, user_id=d312ba6600684240819fd8a3f23ec053, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:23.191882 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e162c3-2b6a-4c41-bf3d-a43eac9bca26 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:23.363309 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1056/2067] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:23 2022] PUT /identity/v3/projects/ba1e5cda15ff461ab99b1c49b3ced4f1/users/d312ba6600684240819fd8a3f23ec053/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:23.376269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301563f-7ec4-43ba-8ae3-1a5004b5c406 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:23.376552 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301563f-7ec4-43ba-8ae3-1a5004b5c406 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:23.376821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301563f-7ec4-43ba-8ae3-1a5004b5c406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:23.413289 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-7301563f-7ec4-43ba-8ae3-1a5004b5c406 None None] MFA Rules not processed for user `d312ba6600684240819fd8a3f23ec053`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:23.415622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-7301563f-7ec4-43ba-8ae3-1a5004b5c406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:23.487601 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1012/2068] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:23.499677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e162c3-2b6a-4c41-bf3d-a43eac9bca26 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:23.501677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-57e162c3-2b6a-4c41-bf3d-a43eac9bca26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:23.568907 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e162c3-2b6a-4c41-bf3d-a43eac9bca26 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:23.574896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e6b881-6b65-4cc4-bfdc-329a0d961583 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:23.575757 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6b881-6b65-4cc4-bfdc-329a0d961583 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:23.576170 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6b881-6b65-4cc4-bfdc-329a0d961583 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:23.576547 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6b881-6b65-4cc4-bfdc-329a0d961583 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:23.580716 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:23.588580 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e6b881-6b65-4cc4-bfdc-329a0d961583 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:23.590426 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e6b881-6b65-4cc4-bfdc-329a0d961583 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:23.600613 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1057/2069] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:23.699975 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:24.946114 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:25.358618 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:26.985033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301563f-7ec4-43ba-8ae3-1a5004b5c406 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:26.987096 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-7301563f-7ec4-43ba-8ae3-1a5004b5c406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:27.117037 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301563f-7ec4-43ba-8ae3-1a5004b5c406 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:27.121160 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90352779-a42c-4327-ad23-42b6f509cc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:27.121808 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90352779-a42c-4327-ad23-42b6f509cc8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:27.122084 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90352779-a42c-4327-ad23-42b6f509cc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:27.122353 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90352779-a42c-4327-ad23-42b6f509cc8d admin admin] PATH_INFO: `/v3/users/45e325a3200b4cd18b15487cc0d28445` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:27.123663 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90352779-a42c-4327-ad23-42b6f509cc8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=45e325a3200b4cd18b15487cc0d28445)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:27.127100 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90352779-a42c-4327-ad23-42b6f509cc8d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:27.366454 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-90352779-a42c-4327-ad23-42b6f509cc8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45e325a3200b4cd18b15487cc0d28445'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:27.376790 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-90352779-a42c-4327-ad23-42b6f509cc8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45e325a3200b4cd18b15487cc0d28445'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:27.377546 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-90352779-a42c-4327-ad23-42b6f509cc8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45e325a3200b4cd18b15487cc0d28445'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:27.393252 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-90352779-a42c-4327-ad23-42b6f509cc8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45e325a3200b4cd18b15487cc0d28445'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:27.393995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-90352779-a42c-4327-ad23-42b6f509cc8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45e325a3200b4cd18b15487cc0d28445'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:27.473190 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1013/2070] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:45:26 2022] DELETE /identity/v3/users/45e325a3200b4cd18b15487cc0d28445 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:45:27.478510 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d794816-5af1-490d-a091-85e3c2dc14af None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:27.479563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d794816-5af1-490d-a091-85e3c2dc14af None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:27.480193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d794816-5af1-490d-a091-85e3c2dc14af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:27.512344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-2d794816-5af1-490d-a091-85e3c2dc14af None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:27.513590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-2d794816-5af1-490d-a091-85e3c2dc14af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:27.592487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1058/2071] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:27.604299 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90352779-a42c-4327-ad23-42b6f509cc8d admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:27.606920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-90352779-a42c-4327-ad23-42b6f509cc8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:27.663900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:27.691885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90352779-a42c-4327-ad23-42b6f509cc8d admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:27.696753 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d5060-a29e-4cb7-b733-3f270161fc2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:27.697445 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d5060-a29e-4cb7-b733-3f270161fc2f service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:27.697801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d5060-a29e-4cb7-b733-3f270161fc2f service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:27.698750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d5060-a29e-4cb7-b733-3f270161fc2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:27.709163 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d5060-a29e-4cb7-b733-3f270161fc2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:27.710828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d5060-a29e-4cb7-b733-3f270161fc2f service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:27.721480 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1014/2072] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:28.199305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d794816-5af1-490d-a091-85e3c2dc14af None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:28.200801 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-2d794816-5af1-490d-a091-85e3c2dc14af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:28.215547 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d794816-5af1-490d-a091-85e3c2dc14af None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:28.221532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f987e71-4cef-45a8-adee-d50a0da5bd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:28.222239 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f987e71-4cef-45a8-adee-d50a0da5bd70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:28.222832 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f987e71-4cef-45a8-adee-d50a0da5bd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:28.225608 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f987e71-4cef-45a8-adee-d50a0da5bd70 admin admin] PATH_INFO: `/v3/projects/6a76043a3cd74df8a53c81b1a77568a3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:28.225608 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f987e71-4cef-45a8-adee-d50a0da5bd70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a76043a3cd74df8a53c81b1a77568a3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:28.226677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f987e71-4cef-45a8-adee-d50a0da5bd70 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:28.395081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-3f987e71-4cef-45a8-adee-d50a0da5bd70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a76043a3cd74df8a53c81b1a77568a3', 'request_id': 'req-3f987e71-4cef-45a8-adee-d50a0da5bd70'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:28.403285 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-3f987e71-4cef-45a8-adee-d50a0da5bd70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a76043a3cd74df8a53c81b1a77568a3', 'request_id': 'req-3f987e71-4cef-45a8-adee-d50a0da5bd70'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:28.421939 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-3f987e71-4cef-45a8-adee-d50a0da5bd70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a76043a3cd74df8a53c81b1a77568a3', 'request_id': 'req-3f987e71-4cef-45a8-adee-d50a0da5bd70'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:28.512644 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-3f987e71-4cef-45a8-adee-d50a0da5bd70 admin admin] The token cache is being invalidate because project 6a76043a3cd74df8a53c81b1a77568a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:45:28.513631 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-3f987e71-4cef-45a8-adee-d50a0da5bd70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:28.514494 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-3f987e71-4cef-45a8-adee-d50a0da5bd70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:28.516467 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1059/2073] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:45:28 2022] DELETE /identity/v3/projects/6a76043a3cd74df8a53c81b1a77568a3 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:28.520750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d5060-a29e-4cb7-b733-3f270161fc2f service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:28.534088 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d5060-a29e-4cb7-b733-3f270161fc2f service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:28.608004 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46e421-1aa9-4103-8b39-abfc724415dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:28.608877 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46e421-1aa9-4103-8b39-abfc724415dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:28.609160 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46e421-1aa9-4103-8b39-abfc724415dc admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:28.609429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46e421-1aa9-4103-8b39-abfc724415dc admin admin] PATH_INFO: `/v3/users/8bfebb76233d4986959b0acb67f501ff` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:28.610620 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e46e421-1aa9-4103-8b39-abfc724415dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bfebb76233d4986959b0acb67f501ff)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:28.612645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e46e421-1aa9-4103-8b39-abfc724415dc admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:28.799791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342909be-77cf-4248-abda-0bf89a7c9844 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:28.800072 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342909be-77cf-4248-abda-0bf89a7c9844 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:28.800381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342909be-77cf-4248-abda-0bf89a7c9844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:28.833068 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-342909be-77cf-4248-abda-0bf89a7c9844 None None] MFA Rules not processed for user `9c2fdfdb62b54f05b7d81f14ec197cca`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:28.834360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-342909be-77cf-4248-abda-0bf89a7c9844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:28.859700 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0e46e421-1aa9-4103-8b39-abfc724415dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bfebb76233d4986959b0acb67f501ff'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:28.867222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0e46e421-1aa9-4103-8b39-abfc724415dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bfebb76233d4986959b0acb67f501ff'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:28.868836 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0e46e421-1aa9-4103-8b39-abfc724415dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bfebb76233d4986959b0acb67f501ff'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:28.888473 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0e46e421-1aa9-4103-8b39-abfc724415dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bfebb76233d4986959b0acb67f501ff'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:28.889377 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0e46e421-1aa9-4103-8b39-abfc724415dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bfebb76233d4986959b0acb67f501ff'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:28.909848 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:28.924467 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1060/2074] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:45:28.939184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342909be-77cf-4248-abda-0bf89a7c9844 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:28.940660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-342909be-77cf-4248-abda-0bf89a7c9844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:29.012601 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1015/2075] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:45:28 2022] DELETE /identity/v3/users/8bfebb76233d4986959b0acb67f501ff => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:29.024539 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342909be-77cf-4248-abda-0bf89a7c9844 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:29.092768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8617c-0350-45ac-8a4f-6f77092d5562 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:29.093405 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8617c-0350-45ac-8a4f-6f77092d5562 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:29.093771 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8617c-0350-45ac-8a4f-6f77092d5562 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:29.094049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8617c-0350-45ac-8a4f-6f77092d5562 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:29.151605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8617c-0350-45ac-8a4f-6f77092d5562 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:29.153329 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8617c-0350-45ac-8a4f-6f77092d5562 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:29.162917 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1061/2076] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:45:29.435016 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46e421-1aa9-4103-8b39-abfc724415dc admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:29.437761 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-0e46e421-1aa9-4103-8b39-abfc724415dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:29.534645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:29.547864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46e421-1aa9-4103-8b39-abfc724415dc admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:29.552676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d21ac-1ca4-475a-871b-12912c1936bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:29.553321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d21ac-1ca4-475a-871b-12912c1936bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:29.553601 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d21ac-1ca4-475a-871b-12912c1936bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:29.553920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d21ac-1ca4-475a-871b-12912c1936bf admin admin] PATH_INFO: `/v3/projects/93212f2703334744988911df7988ad9b` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:29.555081 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d21ac-1ca4-475a-871b-12912c1936bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=93212f2703334744988911df7988ad9b)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:29.557147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d21ac-1ca4-475a-871b-12912c1936bf admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:29.696606 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b81d21ac-1ca4-475a-871b-12912c1936bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93212f2703334744988911df7988ad9b', 'request_id': 'req-b81d21ac-1ca4-475a-871b-12912c1936bf'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:29.802853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b81d21ac-1ca4-475a-871b-12912c1936bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93212f2703334744988911df7988ad9b', 'request_id': 'req-b81d21ac-1ca4-475a-871b-12912c1936bf'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:29.812321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b81d21ac-1ca4-475a-871b-12912c1936bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93212f2703334744988911df7988ad9b', 'request_id': 'req-b81d21ac-1ca4-475a-871b-12912c1936bf'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:29.826848 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b81d21ac-1ca4-475a-871b-12912c1936bf admin admin] The token cache is being invalidate because project 93212f2703334744988911df7988ad9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:45:29.827471 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b81d21ac-1ca4-475a-871b-12912c1936bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:29.828383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b81d21ac-1ca4-475a-871b-12912c1936bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:29.831324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1016/2077] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:45:29 2022] DELETE /identity/v3/projects/93212f2703334744988911df7988ad9b => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:29.834706 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8617c-0350-45ac-8a4f-6f77092d5562 service nova] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:29.847441 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8617c-0350-45ac-8a4f-6f77092d5562 service nova] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:29.944689 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6f2bc-f87f-40e7-b432-febed95e1e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:29.946572 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6f2bc-f87f-40e7-b432-febed95e1e69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:29.947274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6f2bc-f87f-40e7-b432-febed95e1e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:29.947775 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6f2bc-f87f-40e7-b432-febed95e1e69 admin admin] PATH_INFO: `/v3/users/4e04913e6b3c4fe697443c209844f701` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:29.949389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab6f2bc-f87f-40e7-b432-febed95e1e69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e04913e6b3c4fe697443c209844f701)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:29.951727 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab6f2bc-f87f-40e7-b432-febed95e1e69 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:30.212809 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0ab6f2bc-f87f-40e7-b432-febed95e1e69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e04913e6b3c4fe697443c209844f701'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:30.213478 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0ab6f2bc-f87f-40e7-b432-febed95e1e69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e04913e6b3c4fe697443c209844f701'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:30.221108 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0ab6f2bc-f87f-40e7-b432-febed95e1e69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e04913e6b3c4fe697443c209844f701'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:30.317248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0ab6f2bc-f87f-40e7-b432-febed95e1e69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e04913e6b3c4fe697443c209844f701'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:30.319074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0ab6f2bc-f87f-40e7-b432-febed95e1e69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e04913e6b3c4fe697443c209844f701'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:30.342313 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1062/2078] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:45:29 2022] DELETE /identity/v3/users/4e04913e6b3c4fe697443c209844f701 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:30.460196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:30.851064 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d21ac-1ca4-475a-871b-12912c1936bf admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:30.852849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b81d21ac-1ca4-475a-871b-12912c1936bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:30.875671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d21ac-1ca4-475a-871b-12912c1936bf admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:30.986447 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dae513-299d-4307-93ca-0e313edbe796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:30.987495 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dae513-299d-4307-93ca-0e313edbe796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:30.987627 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dae513-299d-4307-93ca-0e313edbe796 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:30.987912 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dae513-299d-4307-93ca-0e313edbe796 admin admin] PATH_INFO: `/v3/projects/a0624f041fa4412dbfe294e5fea71867` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:30.989075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dae513-299d-4307-93ca-0e313edbe796 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0624f041fa4412dbfe294e5fea71867)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:30.991369 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dae513-299d-4307-93ca-0e313edbe796 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:31.107862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b9dae513-299d-4307-93ca-0e313edbe796 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0624f041fa4412dbfe294e5fea71867', 'request_id': 'req-b9dae513-299d-4307-93ca-0e313edbe796'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:31.197317 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b9dae513-299d-4307-93ca-0e313edbe796 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0624f041fa4412dbfe294e5fea71867', 'request_id': 'req-b9dae513-299d-4307-93ca-0e313edbe796'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:31.205072 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b9dae513-299d-4307-93ca-0e313edbe796 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0624f041fa4412dbfe294e5fea71867', 'request_id': 'req-b9dae513-299d-4307-93ca-0e313edbe796'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:31.229657 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b9dae513-299d-4307-93ca-0e313edbe796 admin admin] The token cache is being invalidate because project a0624f041fa4412dbfe294e5fea71867 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:45:31.230193 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b9dae513-299d-4307-93ca-0e313edbe796 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:31.231069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b9dae513-299d-4307-93ca-0e313edbe796 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:31.233836 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1017/2079] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:45:30 2022] DELETE /identity/v3/projects/a0624f041fa4412dbfe294e5fea71867 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:31.238728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9250d6-92c7-4126-8f5c-d5a340a728ee None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:31.239225 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9250d6-92c7-4126-8f5c-d5a340a728ee None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:31.239225 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9250d6-92c7-4126-8f5c-d5a340a728ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:31.262999 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-1d9250d6-92c7-4126-8f5c-d5a340a728ee None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:31.264338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-1d9250d6-92c7-4126-8f5c-d5a340a728ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:31.350939 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1063/2080] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:31.359857 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb1bc3-52eb-4157-8203-258bae82cac2 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:31.361052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb1bc3-52eb-4157-8203-258bae82cac2 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:31.361052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb1bc3-52eb-4157-8203-258bae82cac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:31.392858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-97bb1bc3-52eb-4157-8203-258bae82cac2 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:31.394486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-97bb1bc3-52eb-4157-8203-258bae82cac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:31.404003 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1018/2081] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:31.418585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9250d6-92c7-4126-8f5c-d5a340a728ee None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:31.433336 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9250d6-92c7-4126-8f5c-d5a340a728ee None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:31.438223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590ef7a-8103-4228-a19e-1745d1d9827c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:31.438827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590ef7a-8103-4228-a19e-1745d1d9827c admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:31.439128 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590ef7a-8103-4228-a19e-1745d1d9827c admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:31.439371 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590ef7a-8103-4228-a19e-1745d1d9827c admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:31.440164 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590ef7a-8103-4228-a19e-1745d1d9827c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:45:31.440515 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590ef7a-8103-4228-a19e-1745d1d9827c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:31.442757 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590ef7a-8103-4228-a19e-1745d1d9827c admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:31.466927 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1064/2082] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:45:31.469115 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb1bc3-52eb-4157-8203-258bae82cac2 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:31.483320 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb1bc3-52eb-4157-8203-258bae82cac2 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:31.492932 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22abf1-4a9d-4427-9fbb-c22450075431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:31.495534 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22abf1-4a9d-4427-9fbb-c22450075431 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:31.495983 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22abf1-4a9d-4427-9fbb-c22450075431 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:31.496923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22abf1-4a9d-4427-9fbb-c22450075431 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:31.501408 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22abf1-4a9d-4427-9fbb-c22450075431 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:31.503354 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22abf1-4a9d-4427-9fbb-c22450075431 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:31.549949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:31.693511 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1019/2083] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:45:31 2022] POST /identity/v3/projects => generated 373 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:31.695393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590ef7a-8103-4228-a19e-1745d1d9827c admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:31.710450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590ef7a-8103-4228-a19e-1745d1d9827c admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:31.779175 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:31.819798 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509cea1-ba00-440f-9ec3-67d66ca8166d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:31.824152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509cea1-ba00-440f-9ec3-67d66ca8166d admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:31.824152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509cea1-ba00-440f-9ec3-67d66ca8166d admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:31.824152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509cea1-ba00-440f-9ec3-67d66ca8166d admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:31.824152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509cea1-ba00-440f-9ec3-67d66ca8166d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:31.825020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509cea1-ba00-440f-9ec3-67d66ca8166d admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:31.917427 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1065/2084] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:45:31 2022] POST /identity/v3/users => generated 287 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:31.918718 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22abf1-4a9d-4427-9fbb-c22450075431 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:31.935491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22abf1-4a9d-4427-9fbb-c22450075431 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:31.942375 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8840e-07ea-4272-b1af-c36ce85e7c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:31.943416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8840e-07ea-4272-b1af-c36ce85e7c75 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:31.943919 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8840e-07ea-4272-b1af-c36ce85e7c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:31.945079 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8840e-07ea-4272-b1af-c36ce85e7c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:31.953437 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8840e-07ea-4272-b1af-c36ce85e7c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:31.955078 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8840e-07ea-4272-b1af-c36ce85e7c75 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:31.978395 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1020/2085] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:31.982465 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509cea1-ba00-440f-9ec3-67d66ca8166d admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:32.003260 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509cea1-ba00-440f-9ec3-67d66ca8166d admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:32.010275 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8d26f-7094-4a15-a444-fde76e78760b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:32.011090 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8d26f-7094-4a15-a444-fde76e78760b admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:32.011446 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8d26f-7094-4a15-a444-fde76e78760b admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:32.012048 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8d26f-7094-4a15-a444-fde76e78760b admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:32.012961 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8d26f-7094-4a15-a444-fde76e78760b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:32.015823 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8d26f-7094-4a15-a444-fde76e78760b admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:32.033440 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1066/2086] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:45:32.036306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8840e-07ea-4272-b1af-c36ce85e7c75 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:32.045882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8840e-07ea-4272-b1af-c36ce85e7c75 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:32.051300 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c6451-6f30-49d9-958b-ae3f91d6e918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:32.051973 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c6451-6f30-49d9-958b-ae3f91d6e918 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:32.052444 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c6451-6f30-49d9-958b-ae3f91d6e918 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:32.052910 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c6451-6f30-49d9-958b-ae3f91d6e918 admin admin] PATH_INFO: `/v3/projects/7242514fa6a04b6f8094ef2353652e77/users/345872a1c33f4e28bd44947c23da973b/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:32.076130 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985c6451-6f30-49d9-958b-ae3f91d6e918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7242514fa6a04b6f8094ef2353652e77, user_id=345872a1c33f4e28bd44947c23da973b, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:32.079011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985c6451-6f30-49d9-958b-ae3f91d6e918 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:32.193852 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1021/2087] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:32 2022] PUT /identity/v3/projects/7242514fa6a04b6f8094ef2353652e77/users/345872a1c33f4e28bd44947c23da973b/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:45:32.196023 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ba9b7-253d-4e2d-b25a-5bab43752d86 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:32.196492 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ba9b7-253d-4e2d-b25a-5bab43752d86 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:32.196586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ba9b7-253d-4e2d-b25a-5bab43752d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:32.220449 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660dcb1d-3390-4174-818c-83b4824b2f5f None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:32.221312 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660dcb1d-3390-4174-818c-83b4824b2f5f None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:32.222329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660dcb1d-3390-4174-818c-83b4824b2f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:32.247512 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-f82ba9b7-253d-4e2d-b25a-5bab43752d86 None None] MFA Rules not processed for user `345872a1c33f4e28bd44947c23da973b`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:32.250624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f82ba9b7-253d-4e2d-b25a-5bab43752d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:32.253126 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-660dcb1d-3390-4174-818c-83b4824b2f5f None None] MFA Rules not processed for user `9c2fdfdb62b54f05b7d81f14ec197cca`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:32.254350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-660dcb1d-3390-4174-818c-83b4824b2f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:32.366221 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1022/2088] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:45:32.389012 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1067/2089] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:32.452587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660dcb1d-3390-4174-818c-83b4824b2f5f None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:32.457405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-660dcb1d-3390-4174-818c-83b4824b2f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:32.562604 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660dcb1d-3390-4174-818c-83b4824b2f5f None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:32.569949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7879f28-76ca-4164-9055-311326942537 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:32.570608 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7879f28-76ca-4164-9055-311326942537 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:32.570888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7879f28-76ca-4164-9055-311326942537 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:32.571215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7879f28-76ca-4164-9055-311326942537 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:32.583101 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7879f28-76ca-4164-9055-311326942537 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:32.584838 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7879f28-76ca-4164-9055-311326942537 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:32.598300 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1023/2090] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:32.640513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ba9b7-253d-4e2d-b25a-5bab43752d86 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:32.761798 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ba9b7-253d-4e2d-b25a-5bab43752d86 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:32.766845 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef527a-b2cc-48c8-ace1-7e03487e5851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:32.772272 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef527a-b2cc-48c8-ace1-7e03487e5851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:32.772272 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef527a-b2cc-48c8-ace1-7e03487e5851 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:32.772272 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef527a-b2cc-48c8-ace1-7e03487e5851 admin admin] PATH_INFO: `/v3/users/345872a1c33f4e28bd44947c23da973b` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:32.772272 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef527a-b2cc-48c8-ace1-7e03487e5851 admin admin] RBAC: Authorizing `identity:delete_user(user_id=345872a1c33f4e28bd44947c23da973b)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:32.772272 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef527a-b2cc-48c8-ace1-7e03487e5851 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:33.051361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-72ef527a-b2cc-48c8-ace1-7e03487e5851 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '345872a1c33f4e28bd44947c23da973b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:33.055451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-72ef527a-b2cc-48c8-ace1-7e03487e5851 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '345872a1c33f4e28bd44947c23da973b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:33.074517 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-72ef527a-b2cc-48c8-ace1-7e03487e5851 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '345872a1c33f4e28bd44947c23da973b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:33.197312 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-72ef527a-b2cc-48c8-ace1-7e03487e5851 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '345872a1c33f4e28bd44947c23da973b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:33.197976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-72ef527a-b2cc-48c8-ace1-7e03487e5851 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '345872a1c33f4e28bd44947c23da973b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:33.225578 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1068/2091] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:45:32 2022] DELETE /identity/v3/users/345872a1c33f4e28bd44947c23da973b => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:33.227645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57533b5-9661-4eb6-989f-8d2ff8192114 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:33.228109 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57533b5-9661-4eb6-989f-8d2ff8192114 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:33.228248 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57533b5-9661-4eb6-989f-8d2ff8192114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:33.249061 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-f57533b5-9661-4eb6-989f-8d2ff8192114 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:33.250529 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f57533b5-9661-4eb6-989f-8d2ff8192114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:33.319632 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1024/2092] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:45:33.330321 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef527a-b2cc-48c8-ace1-7e03487e5851 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:33.332371 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-72ef527a-b2cc-48c8-ace1-7e03487e5851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:33.399356 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef527a-b2cc-48c8-ace1-7e03487e5851 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:33.403081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e7359-cb99-45e0-a167-dc466f7e3e5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:33.403681 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e7359-cb99-45e0-a167-dc466f7e3e5e service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:33.403955 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e7359-cb99-45e0-a167-dc466f7e3e5e service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:33.404251 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e7359-cb99-45e0-a167-dc466f7e3e5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:33.417929 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e7359-cb99-45e0-a167-dc466f7e3e5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:33.417929 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e7359-cb99-45e0-a167-dc466f7e3e5e service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:33.426922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1069/2093] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:45:33.690869 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:33.763930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:33.976488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57533b5-9661-4eb6-989f-8d2ff8192114 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:33.978833 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f57533b5-9661-4eb6-989f-8d2ff8192114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:34.001599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57533b5-9661-4eb6-989f-8d2ff8192114 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:34.009937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013a2c0-e450-46ff-a687-b6bf61b23650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:34.009937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013a2c0-e450-46ff-a687-b6bf61b23650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:34.009937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013a2c0-e450-46ff-a687-b6bf61b23650 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:34.009937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013a2c0-e450-46ff-a687-b6bf61b23650 admin admin] PATH_INFO: `/v3/projects/7242514fa6a04b6f8094ef2353652e77` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:34.010956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013a2c0-e450-46ff-a687-b6bf61b23650 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7242514fa6a04b6f8094ef2353652e77)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:34.012908 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013a2c0-e450-46ff-a687-b6bf61b23650 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:34.166396 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5013a2c0-e450-46ff-a687-b6bf61b23650 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7242514fa6a04b6f8094ef2353652e77', 'request_id': 'req-5013a2c0-e450-46ff-a687-b6bf61b23650'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:34.272818 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5013a2c0-e450-46ff-a687-b6bf61b23650 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7242514fa6a04b6f8094ef2353652e77', 'request_id': 'req-5013a2c0-e450-46ff-a687-b6bf61b23650'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:34.281660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5013a2c0-e450-46ff-a687-b6bf61b23650 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7242514fa6a04b6f8094ef2353652e77', 'request_id': 'req-5013a2c0-e450-46ff-a687-b6bf61b23650'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:34.295897 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5013a2c0-e450-46ff-a687-b6bf61b23650 admin admin] The token cache is being invalidate because project 7242514fa6a04b6f8094ef2353652e77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:45:34.296217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5013a2c0-e450-46ff-a687-b6bf61b23650 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:34.296891 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5013a2c0-e450-46ff-a687-b6bf61b23650 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:34.298464 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1025/2094] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:45:33 2022] DELETE /identity/v3/projects/7242514fa6a04b6f8094ef2353652e77 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:34.311531 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b082e15-faf5-4998-b37d-9e35b2d47f78 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:34.311531 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b082e15-faf5-4998-b37d-9e35b2d47f78 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:34.311531 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b082e15-faf5-4998-b37d-9e35b2d47f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:34.326917 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-8b082e15-faf5-4998-b37d-9e35b2d47f78 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:34.328055 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8b082e15-faf5-4998-b37d-9e35b2d47f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:34.393669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1070/2095] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:34.403889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd0ecf-b133-4466-82e2-b566e7d77518 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:34.404236 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd0ecf-b133-4466-82e2-b566e7d77518 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:34.404454 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd0ecf-b133-4466-82e2-b566e7d77518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:34.428430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-23dd0ecf-b133-4466-82e2-b566e7d77518 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:34.429798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-23dd0ecf-b133-4466-82e2-b566e7d77518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:34.438403 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1026/2096] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:45:34.444754 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b082e15-faf5-4998-b37d-9e35b2d47f78 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:34.456779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b082e15-faf5-4998-b37d-9e35b2d47f78 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:34.458661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5eb2a2-83f4-4964-9faf-76de33416858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:34.459263 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5eb2a2-83f4-4964-9faf-76de33416858 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:34.459536 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5eb2a2-83f4-4964-9faf-76de33416858 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:34.459801 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5eb2a2-83f4-4964-9faf-76de33416858 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:34.460476 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5eb2a2-83f4-4964-9faf-76de33416858 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:45:34.460774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5eb2a2-83f4-4964-9faf-76de33416858 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:34.463031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5eb2a2-83f4-4964-9faf-76de33416858 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:34.475216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1071/2097] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:45:34.480411 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd0ecf-b133-4466-82e2-b566e7d77518 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:34.490010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd0ecf-b133-4466-82e2-b566e7d77518 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:34.495142 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93216c45-a03b-4ba1-a687-0ac2e597be0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:34.495752 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93216c45-a03b-4ba1-a687-0ac2e597be0e admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:34.496046 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93216c45-a03b-4ba1-a687-0ac2e597be0e admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:34.496354 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93216c45-a03b-4ba1-a687-0ac2e597be0e admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:34.497458 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93216c45-a03b-4ba1-a687-0ac2e597be0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:34.499182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93216c45-a03b-4ba1-a687-0ac2e597be0e admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:34.610147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1027/2098] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:45:34 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:45:34.614482 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5eb2a2-83f4-4964-9faf-76de33416858 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:34.625224 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5eb2a2-83f4-4964-9faf-76de33416858 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:34.686049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe8051-367f-4994-a520-aa321f28d7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:34.686663 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe8051-367f-4994-a520-aa321f28d7a2 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:34.686939 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe8051-367f-4994-a520-aa321f28d7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:34.687207 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe8051-367f-4994-a520-aa321f28d7a2 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:34.687910 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe8051-367f-4994-a520-aa321f28d7a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:34.689847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe8051-367f-4994-a520-aa321f28d7a2 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:34.860868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1072/2099] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:45:34 2022] POST /identity/v3/users => generated 289 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:34.865079 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93216c45-a03b-4ba1-a687-0ac2e597be0e admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:34.877989 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93216c45-a03b-4ba1-a687-0ac2e597be0e admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:34.884160 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509f7ef-0943-41b8-bf95-bbb1ebbedf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:34.884875 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509f7ef-0943-41b8-bf95-bbb1ebbedf88 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:34.885157 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509f7ef-0943-41b8-bf95-bbb1ebbedf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:34.885429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509f7ef-0943-41b8-bf95-bbb1ebbedf88 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:34.886068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509f7ef-0943-41b8-bf95-bbb1ebbedf88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:34.887676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509f7ef-0943-41b8-bf95-bbb1ebbedf88 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:34.898322 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1028/2100] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:34.907932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe8051-367f-4994-a520-aa321f28d7a2 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:34.918229 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe8051-367f-4994-a520-aa321f28d7a2 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:34.923821 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7627c-bf72-418a-87c6-62b26a32a164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:34.924221 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7627c-bf72-418a-87c6-62b26a32a164 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:34.924609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7627c-bf72-418a-87c6-62b26a32a164 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:34.924949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7627c-bf72-418a-87c6-62b26a32a164 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:34.925667 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7627c-bf72-418a-87c6-62b26a32a164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:34.927378 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7627c-bf72-418a-87c6-62b26a32a164 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:34.939464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1073/2101] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:34.954416 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509f7ef-0943-41b8-bf95-bbb1ebbedf88 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:34.965260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509f7ef-0943-41b8-bf95-bbb1ebbedf88 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:34.970063 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059abe9-fc7c-4fa0-8328-738394a08446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:34.970715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059abe9-fc7c-4fa0-8328-738394a08446 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:34.970988 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059abe9-fc7c-4fa0-8328-738394a08446 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:34.971257 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059abe9-fc7c-4fa0-8328-738394a08446 admin admin] PATH_INFO: `/v3/projects/f42dbdfe1c064b2080b48f8d40a4638b/users/21feac769b4b48508163f535cdedfbe5/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:34.986009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059abe9-fc7c-4fa0-8328-738394a08446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f42dbdfe1c064b2080b48f8d40a4638b, user_id=21feac769b4b48508163f535cdedfbe5, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:34.988676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059abe9-fc7c-4fa0-8328-738394a08446 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:35.143175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1029/2102] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:34 2022] PUT /identity/v3/projects/f42dbdfe1c064b2080b48f8d40a4638b/users/21feac769b4b48508163f535cdedfbe5/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:45:35.148026 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fb07e-b4c7-4709-a1db-ce80c6899ffe None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:35.148355 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fb07e-b4c7-4709-a1db-ce80c6899ffe None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:35.148628 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fb07e-b4c7-4709-a1db-ce80c6899ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:35.169154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-992fb07e-b4c7-4709-a1db-ce80c6899ffe None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:35.170931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-992fb07e-b4c7-4709-a1db-ce80c6899ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:35.232281 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1074/2103] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:35.242261 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059abe9-fc7c-4fa0-8328-738394a08446 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:35.245058 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3059abe9-fc7c-4fa0-8328-738394a08446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:35.308912 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059abe9-fc7c-4fa0-8328-738394a08446 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:35.313127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6fd3f-02e7-4b22-bccb-8b9af8f4082f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:35.313730 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6fd3f-02e7-4b22-bccb-8b9af8f4082f service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:35.314120 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6fd3f-02e7-4b22-bccb-8b9af8f4082f service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:35.314290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6fd3f-02e7-4b22-bccb-8b9af8f4082f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:35.324397 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6fd3f-02e7-4b22-bccb-8b9af8f4082f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:35.325969 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6fd3f-02e7-4b22-bccb-8b9af8f4082f service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:35.338554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1030/2104] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:35.800684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992fb07e-b4c7-4709-a1db-ce80c6899ffe None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:35.803691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-992fb07e-b4c7-4709-a1db-ce80c6899ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:35.817457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992fb07e-b4c7-4709-a1db-ce80c6899ffe None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:35.822300 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab0899-361b-4793-835d-55eb48e87813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:35.823040 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab0899-361b-4793-835d-55eb48e87813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:35.824049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab0899-361b-4793-835d-55eb48e87813 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:35.824049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab0899-361b-4793-835d-55eb48e87813 admin admin] PATH_INFO: `/v3/users/d312ba6600684240819fd8a3f23ec053` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:35.825376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab0899-361b-4793-835d-55eb48e87813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d312ba6600684240819fd8a3f23ec053)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:35.827458 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab0899-361b-4793-835d-55eb48e87813 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:36.044560 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-f6ab0899-361b-4793-835d-55eb48e87813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd312ba6600684240819fd8a3f23ec053'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:36.045410 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-f6ab0899-361b-4793-835d-55eb48e87813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd312ba6600684240819fd8a3f23ec053'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:36.053819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-f6ab0899-361b-4793-835d-55eb48e87813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd312ba6600684240819fd8a3f23ec053'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:36.122456 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-f6ab0899-361b-4793-835d-55eb48e87813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd312ba6600684240819fd8a3f23ec053'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:36.123083 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-f6ab0899-361b-4793-835d-55eb48e87813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd312ba6600684240819fd8a3f23ec053'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:36.142717 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1075/2105] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:45:35 2022] DELETE /identity/v3/users/d312ba6600684240819fd8a3f23ec053 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:36.147422 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe90696-630a-44d4-a766-df72e4f8ad8e None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:36.147710 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe90696-630a-44d4-a766-df72e4f8ad8e None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:36.147984 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe90696-630a-44d4-a766-df72e4f8ad8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:36.162659 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:36.169050 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-dbe90696-630a-44d4-a766-df72e4f8ad8e None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:36.170389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-dbe90696-630a-44d4-a766-df72e4f8ad8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:36.184406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:36.232331 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1031/2106] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:36.248188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab0899-361b-4793-835d-55eb48e87813 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:36.249667 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f6ab0899-361b-4793-835d-55eb48e87813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:36.312465 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab0899-361b-4793-835d-55eb48e87813 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:36.316112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770118a6-7a8b-442d-b22d-9d0489abf469 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:36.316747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770118a6-7a8b-442d-b22d-9d0489abf469 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:36.317020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770118a6-7a8b-442d-b22d-9d0489abf469 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:36.317285 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770118a6-7a8b-442d-b22d-9d0489abf469 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:36.327170 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770118a6-7a8b-442d-b22d-9d0489abf469 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:36.328658 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770118a6-7a8b-442d-b22d-9d0489abf469 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:36.339247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1076/2107] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:45:36.919005 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe90696-630a-44d4-a766-df72e4f8ad8e None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:36.925046 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-dbe90696-630a-44d4-a766-df72e4f8ad8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:36.947009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe90696-630a-44d4-a766-df72e4f8ad8e None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:36.952574 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504fc7d-781d-498f-9a9d-9d3214e5506d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:36.953463 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504fc7d-781d-498f-9a9d-9d3214e5506d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:36.953737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504fc7d-781d-498f-9a9d-9d3214e5506d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:36.954326 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504fc7d-781d-498f-9a9d-9d3214e5506d admin admin] PATH_INFO: `/v3/projects/ba1e5cda15ff461ab99b1c49b3ced4f1` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:36.956338 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504fc7d-781d-498f-9a9d-9d3214e5506d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba1e5cda15ff461ab99b1c49b3ced4f1)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:36.958307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504fc7d-781d-498f-9a9d-9d3214e5506d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:37.097707 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e504fc7d-781d-498f-9a9d-9d3214e5506d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba1e5cda15ff461ab99b1c49b3ced4f1', 'request_id': 'req-e504fc7d-781d-498f-9a9d-9d3214e5506d'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:37.191248 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e504fc7d-781d-498f-9a9d-9d3214e5506d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba1e5cda15ff461ab99b1c49b3ced4f1', 'request_id': 'req-e504fc7d-781d-498f-9a9d-9d3214e5506d'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:37.198762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e504fc7d-781d-498f-9a9d-9d3214e5506d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba1e5cda15ff461ab99b1c49b3ced4f1', 'request_id': 'req-e504fc7d-781d-498f-9a9d-9d3214e5506d'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:37.216321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e504fc7d-781d-498f-9a9d-9d3214e5506d admin admin] The token cache is being invalidate because project ba1e5cda15ff461ab99b1c49b3ced4f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:45:37.216617 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e504fc7d-781d-498f-9a9d-9d3214e5506d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:37.217441 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e504fc7d-781d-498f-9a9d-9d3214e5506d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:37.220087 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1032/2108] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:45:36 2022] DELETE /identity/v3/projects/ba1e5cda15ff461ab99b1c49b3ced4f1 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:37.224765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf954488-c21d-47e4-80ed-19f26d469816 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:37.225613 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf954488-c21d-47e4-80ed-19f26d469816 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:37.225945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf954488-c21d-47e4-80ed-19f26d469816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:37.248391 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-bf954488-c21d-47e4-80ed-19f26d469816 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:37.250027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-bf954488-c21d-47e4-80ed-19f26d469816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:37.345162 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1077/2109] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:37.346873 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fc705-fce9-42e2-b4f6-6f6b5ce658dd None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:37.347152 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fc705-fce9-42e2-b4f6-6f6b5ce658dd None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:37.347420 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fc705-fce9-42e2-b4f6-6f6b5ce658dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:37.370520 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-eb8fc705-fce9-42e2-b4f6-6f6b5ce658dd None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:37.371730 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-eb8fc705-fce9-42e2-b4f6-6f6b5ce658dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:37.379858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1033/2110] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:37.385533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf954488-c21d-47e4-80ed-19f26d469816 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:37.395833 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf954488-c21d-47e4-80ed-19f26d469816 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:37.401567 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546602b8-64ff-4111-955c-3948a00a4cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:37.402193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546602b8-64ff-4111-955c-3948a00a4cdb admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:37.402444 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546602b8-64ff-4111-955c-3948a00a4cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:37.402712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546602b8-64ff-4111-955c-3948a00a4cdb admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:37.403347 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546602b8-64ff-4111-955c-3948a00a4cdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:45:37.403638 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546602b8-64ff-4111-955c-3948a00a4cdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:37.405256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546602b8-64ff-4111-955c-3948a00a4cdb admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:37.418518 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1078/2111] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:45:37.423155 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fc705-fce9-42e2-b4f6-6f6b5ce658dd None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:37.433787 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fc705-fce9-42e2-b4f6-6f6b5ce658dd None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:37.438661 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0f556-76c3-4a5d-afe0-13ed05929e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:37.439278 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0f556-76c3-4a5d-afe0-13ed05929e37 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:37.439554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0f556-76c3-4a5d-afe0-13ed05929e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:37.439824 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0f556-76c3-4a5d-afe0-13ed05929e37 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:37.440497 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0f556-76c3-4a5d-afe0-13ed05929e37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:37.442397 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0f556-76c3-4a5d-afe0-13ed05929e37 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:37.443286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:37.542178 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1034/2112] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:45:37 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:37.546699 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546602b8-64ff-4111-955c-3948a00a4cdb admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:37.557375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546602b8-64ff-4111-955c-3948a00a4cdb admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:37.628272 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e289e-0b8a-48b8-ad14-e90225ff7698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:37.629320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e289e-0b8a-48b8-ad14-e90225ff7698 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:37.629678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e289e-0b8a-48b8-ad14-e90225ff7698 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:37.630106 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e289e-0b8a-48b8-ad14-e90225ff7698 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:37.630760 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e289e-0b8a-48b8-ad14-e90225ff7698 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:37.632760 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e289e-0b8a-48b8-ad14-e90225ff7698 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:37.752616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1079/2113] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:45:37 2022] POST /identity/v3/users => generated 288 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:37.757242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0f556-76c3-4a5d-afe0-13ed05929e37 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:37.776992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0f556-76c3-4a5d-afe0-13ed05929e37 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:37.782491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97723991-f083-4e48-9ed0-0ffc8c6b4d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:37.783237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97723991-f083-4e48-9ed0-0ffc8c6b4d8d admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:37.783591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97723991-f083-4e48-9ed0-0ffc8c6b4d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:37.783967 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97723991-f083-4e48-9ed0-0ffc8c6b4d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:37.784713 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97723991-f083-4e48-9ed0-0ffc8c6b4d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:37.786512 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97723991-f083-4e48-9ed0-0ffc8c6b4d8d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:37.802010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1035/2114] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:45:37.802962 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e289e-0b8a-48b8-ad14-e90225ff7698 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:37.814517 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e289e-0b8a-48b8-ad14-e90225ff7698 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:37.819287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4ab0c-225f-4e6a-a952-65a3f2283d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:37.819889 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4ab0c-225f-4e6a-a952-65a3f2283d73 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:37.820304 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4ab0c-225f-4e6a-a952-65a3f2283d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:37.820875 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4ab0c-225f-4e6a-a952-65a3f2283d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:37.821375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4ab0c-225f-4e6a-a952-65a3f2283d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:37.822969 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4ab0c-225f-4e6a-a952-65a3f2283d73 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:37.834621 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1080/2115] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:45:37.839892 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97723991-f083-4e48-9ed0-0ffc8c6b4d8d admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:37.850829 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97723991-f083-4e48-9ed0-0ffc8c6b4d8d admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:37.857119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce786b-39cd-4a3a-a6d7-e1c2c08b695d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:37.857855 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce786b-39cd-4a3a-a6d7-e1c2c08b695d admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:37.858307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce786b-39cd-4a3a-a6d7-e1c2c08b695d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:37.858537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce786b-39cd-4a3a-a6d7-e1c2c08b695d admin admin] PATH_INFO: `/v3/projects/27cbee4b5c514b0283717305606258dd/users/99a53a291a1d4c4d93cbd0a22a01a0b3/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:37.877183 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce786b-39cd-4a3a-a6d7-e1c2c08b695d admin admin] RBAC: Authorizing `identity:create_grant(project_id=27cbee4b5c514b0283717305606258dd, user_id=99a53a291a1d4c4d93cbd0a22a01a0b3, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:37.880262 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce786b-39cd-4a3a-a6d7-e1c2c08b695d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:37.987573 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1036/2116] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:37 2022] PUT /identity/v3/projects/27cbee4b5c514b0283717305606258dd/users/99a53a291a1d4c4d93cbd0a22a01a0b3/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:37.999366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830a7ba-d579-46fe-89ce-2b77f9c7d776 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:37.999891 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830a7ba-d579-46fe-89ce-2b77f9c7d776 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:38.000277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830a7ba-d579-46fe-89ce-2b77f9c7d776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:38.032003 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-8830a7ba-d579-46fe-89ce-2b77f9c7d776 None None] MFA Rules not processed for user `99a53a291a1d4c4d93cbd0a22a01a0b3`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:38.033429 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8830a7ba-d579-46fe-89ce-2b77f9c7d776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:38.237661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1081/2117] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:38.250751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce786b-39cd-4a3a-a6d7-e1c2c08b695d admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:38.252685 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a6ce786b-39cd-4a3a-a6d7-e1c2c08b695d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:38.308231 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:38.324874 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce786b-39cd-4a3a-a6d7-e1c2c08b695d admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:38.329230 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f397c0-b193-43ff-b8ba-cf68cd87d52f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:38.329919 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f397c0-b193-43ff-b8ba-cf68cd87d52f service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:38.330401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f397c0-b193-43ff-b8ba-cf68cd87d52f service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:38.330993 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f397c0-b193-43ff-b8ba-cf68cd87d52f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:38.342964 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f397c0-b193-43ff-b8ba-cf68cd87d52f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:38.344576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f397c0-b193-43ff-b8ba-cf68cd87d52f service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:38.354618 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1037/2118] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:38.819926 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:39.564040 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830a7ba-d579-46fe-89ce-2b77f9c7d776 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:39.568637 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8830a7ba-d579-46fe-89ce-2b77f9c7d776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:39.718386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830a7ba-d579-46fe-89ce-2b77f9c7d776 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:39.729754 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9b570-1af6-429d-97a7-ba5c3475e1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:39.730787 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9b570-1af6-429d-97a7-ba5c3475e1e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:39.731194 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9b570-1af6-429d-97a7-ba5c3475e1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:39.733046 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9b570-1af6-429d-97a7-ba5c3475e1e9 admin admin] PATH_INFO: `/v3/users/ab35de2c5ea84103b3e4e08aeeb3f857` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:39.734439 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb9b570-1af6-429d-97a7-ba5c3475e1e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab35de2c5ea84103b3e4e08aeeb3f857)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:39.736410 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb9b570-1af6-429d-97a7-ba5c3475e1e9 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:39.972182 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ccb9b570-1af6-429d-97a7-ba5c3475e1e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab35de2c5ea84103b3e4e08aeeb3f857'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:39.973836 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ccb9b570-1af6-429d-97a7-ba5c3475e1e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab35de2c5ea84103b3e4e08aeeb3f857'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:40.003000 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ccb9b570-1af6-429d-97a7-ba5c3475e1e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab35de2c5ea84103b3e4e08aeeb3f857'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:40.104398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ccb9b570-1af6-429d-97a7-ba5c3475e1e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab35de2c5ea84103b3e4e08aeeb3f857'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:40.109025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ccb9b570-1af6-429d-97a7-ba5c3475e1e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab35de2c5ea84103b3e4e08aeeb3f857'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:40.133485 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1082/2119] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:45:39 2022] DELETE /identity/v3/users/ab35de2c5ea84103b3e4e08aeeb3f857 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:40.135646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df2fe9-0dbf-4921-9027-fabc067baa30 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:40.136074 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df2fe9-0dbf-4921-9027-fabc067baa30 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:40.136567 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df2fe9-0dbf-4921-9027-fabc067baa30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:40.167742 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-57df2fe9-0dbf-4921-9027-fabc067baa30 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:40.169677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-57df2fe9-0dbf-4921-9027-fabc067baa30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:40.275706 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1038/2120] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:40.285208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9b570-1af6-429d-97a7-ba5c3475e1e9 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:40.286857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-ccb9b570-1af6-429d-97a7-ba5c3475e1e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:40.387678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9b570-1af6-429d-97a7-ba5c3475e1e9 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:40.391567 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f20b3-73b9-417e-8bf0-0fdbcdc0d629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:40.409011 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f20b3-73b9-417e-8bf0-0fdbcdc0d629 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:40.409011 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f20b3-73b9-417e-8bf0-0fdbcdc0d629 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:40.409011 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f20b3-73b9-417e-8bf0-0fdbcdc0d629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:40.409011 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f20b3-73b9-417e-8bf0-0fdbcdc0d629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:40.409011 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f20b3-73b9-417e-8bf0-0fdbcdc0d629 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:40.414446 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:40.421660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1083/2121] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:41.864263 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df2fe9-0dbf-4921-9027-fabc067baa30 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:41.866280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-57df2fe9-0dbf-4921-9027-fabc067baa30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:41.885880 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df2fe9-0dbf-4921-9027-fabc067baa30 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:41.893543 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5b796-9d1c-4396-8436-0377a21b3485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:41.894454 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5b796-9d1c-4396-8436-0377a21b3485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:41.894882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5b796-9d1c-4396-8436-0377a21b3485 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:41.895283 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5b796-9d1c-4396-8436-0377a21b3485 admin admin] PATH_INFO: `/v3/projects/f1ca3c31fa114ea5a6628c510e5de3fd` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:41.897299 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5b796-9d1c-4396-8436-0377a21b3485 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1ca3c31fa114ea5a6628c510e5de3fd)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:41.903461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5b796-9d1c-4396-8436-0377a21b3485 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:42.072630 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e0a5b796-9d1c-4396-8436-0377a21b3485 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1ca3c31fa114ea5a6628c510e5de3fd', 'request_id': 'req-e0a5b796-9d1c-4396-8436-0377a21b3485'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:42.198838 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e0a5b796-9d1c-4396-8436-0377a21b3485 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1ca3c31fa114ea5a6628c510e5de3fd', 'request_id': 'req-e0a5b796-9d1c-4396-8436-0377a21b3485'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:42.210249 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e0a5b796-9d1c-4396-8436-0377a21b3485 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1ca3c31fa114ea5a6628c510e5de3fd', 'request_id': 'req-e0a5b796-9d1c-4396-8436-0377a21b3485'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:42.280411 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e0a5b796-9d1c-4396-8436-0377a21b3485 admin admin] The token cache is being invalidate because project f1ca3c31fa114ea5a6628c510e5de3fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:45:42.281763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e0a5b796-9d1c-4396-8436-0377a21b3485 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:42.292123 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e0a5b796-9d1c-4396-8436-0377a21b3485 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:42.299408 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1039/2122] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:45:41 2022] DELETE /identity/v3/projects/f1ca3c31fa114ea5a6628c510e5de3fd => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:42.305152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1167d9-61db-4521-9078-328264abda9f None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:42.305152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1167d9-61db-4521-9078-328264abda9f None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:42.305677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1167d9-61db-4521-9078-328264abda9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:42.330651 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-dc1167d9-61db-4521-9078-328264abda9f None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:42.332374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-dc1167d9-61db-4521-9078-328264abda9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:42.454003 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cc176-6242-4177-b9bd-10b0c2752c53 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:42.454701 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cc176-6242-4177-b9bd-10b0c2752c53 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:42.455195 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cc176-6242-4177-b9bd-10b0c2752c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:42.467878 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1084/2123] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:42.541081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1167d9-61db-4521-9078-328264abda9f None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:42.543004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-dc1167d9-61db-4521-9078-328264abda9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:42.546006 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-f65cc176-6242-4177-b9bd-10b0c2752c53 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:42.548624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f65cc176-6242-4177-b9bd-10b0c2752c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:42.591365 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1040/2124] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:42.591628 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1167d9-61db-4521-9078-328264abda9f None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:42.593114 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cc176-6242-4177-b9bd-10b0c2752c53 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:42.609703 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b3f63-1848-498d-9821-1062242da8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:42.610653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b3f63-1848-498d-9821-1062242da8ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:42.611184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b3f63-1848-498d-9821-1062242da8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:42.612897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b3f63-1848-498d-9821-1062242da8ad admin admin] PATH_INFO: `/v3/users/c777615176614017b08f95a6d5fdc421` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:42.614861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b3f63-1848-498d-9821-1062242da8ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=c777615176614017b08f95a6d5fdc421)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:42.617324 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b3f63-1848-498d-9821-1062242da8ad admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:42.622705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cc176-6242-4177-b9bd-10b0c2752c53 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:42.642513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedfa9e-d2c0-4d51-8fa0-fc70c2acc16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:42.643328 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedfa9e-d2c0-4d51-8fa0-fc70c2acc16b admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:42.651683 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedfa9e-d2c0-4d51-8fa0-fc70c2acc16b admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:42.651683 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedfa9e-d2c0-4d51-8fa0-fc70c2acc16b admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:42.651683 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedfa9e-d2c0-4d51-8fa0-fc70c2acc16b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:45:42.655753 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedfa9e-d2c0-4d51-8fa0-fc70c2acc16b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:42.657551 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedfa9e-d2c0-4d51-8fa0-fc70c2acc16b admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:42.698473 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1041/2125] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:45:42.720828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedfa9e-d2c0-4d51-8fa0-fc70c2acc16b admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:42.740590 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedfa9e-d2c0-4d51-8fa0-fc70c2acc16b admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:42.754526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caae0ad-354c-4588-9c41-f44bc7705db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:42.755254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caae0ad-354c-4588-9c41-f44bc7705db1 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:42.755475 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caae0ad-354c-4588-9c41-f44bc7705db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:42.755767 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caae0ad-354c-4588-9c41-f44bc7705db1 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:42.756445 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caae0ad-354c-4588-9c41-f44bc7705db1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:42.759103 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caae0ad-354c-4588-9c41-f44bc7705db1 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:42.904554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1042/2126] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:45:42 2022] POST /identity/v3/projects => generated 375 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:42.904554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582fb6f-11f2-4754-8b51-b7c204408a59 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:42.905307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582fb6f-11f2-4754-8b51-b7c204408a59 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:42.905307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582fb6f-11f2-4754-8b51-b7c204408a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:42.953197 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-5582fb6f-11f2-4754-8b51-b7c204408a59 None None] MFA Rules not processed for user `21feac769b4b48508163f535cdedfbe5`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:42.956686 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-5582fb6f-11f2-4754-8b51-b7c204408a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:42.972241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:42.987134 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b17b3f63-1848-498d-9821-1062242da8ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c777615176614017b08f95a6d5fdc421'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:42.987914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b17b3f63-1848-498d-9821-1062242da8ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c777615176614017b08f95a6d5fdc421'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:42.999940 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b17b3f63-1848-498d-9821-1062242da8ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c777615176614017b08f95a6d5fdc421'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:43.055186 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1043/2127] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:43.059127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582fb6f-11f2-4754-8b51-b7c204408a59 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:43.059127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-5582fb6f-11f2-4754-8b51-b7c204408a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:43.123759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b17b3f63-1848-498d-9821-1062242da8ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c777615176614017b08f95a6d5fdc421'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:43.124744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b17b3f63-1848-498d-9821-1062242da8ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c777615176614017b08f95a6d5fdc421'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:43.159505 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582fb6f-11f2-4754-8b51-b7c204408a59 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:43.166987 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6dc3b-ad10-4345-a542-9f615a71af0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:43.168095 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6dc3b-ad10-4345-a542-9f615a71af0e admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:43.168553 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6dc3b-ad10-4345-a542-9f615a71af0e admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:43.168941 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6dc3b-ad10-4345-a542-9f615a71af0e admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:43.170119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6dc3b-ad10-4345-a542-9f615a71af0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:43.172789 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6dc3b-ad10-4345-a542-9f615a71af0e admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:43.180678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1085/2128] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:45:42 2022] DELETE /identity/v3/users/c777615176614017b08f95a6d5fdc421 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:45:43.180678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b3f63-1848-498d-9821-1062242da8ad admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:43.180962 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b17b3f63-1848-498d-9821-1062242da8ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:43.307787 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1044/2129] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:45:43 2022] POST /identity/v3/users => generated 288 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:43.313290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c04ca-46ed-42f0-addf-6f61a212a1cc None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:43.314404 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c04ca-46ed-42f0-addf-6f61a212a1cc None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:43.315893 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c04ca-46ed-42f0-addf-6f61a212a1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:43.350539 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b3f63-1848-498d-9821-1062242da8ad admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:43.360995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fa720-53a9-4211-8f91-6103f4311178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:43.368611 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fa720-53a9-4211-8f91-6103f4311178 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:43.369312 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fa720-53a9-4211-8f91-6103f4311178 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:43.370744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fa720-53a9-4211-8f91-6103f4311178 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:43.371530 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7fa720-53a9-4211-8f91-6103f4311178 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:43.375721 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7fa720-53a9-4211-8f91-6103f4311178 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:43.384354 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-a42c04ca-46ed-42f0-addf-6f61a212a1cc None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:43.385596 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a42c04ca-46ed-42f0-addf-6f61a212a1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:43.394020 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1045/2130] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:43.394860 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c04ca-46ed-42f0-addf-6f61a212a1cc None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:43.412290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c04ca-46ed-42f0-addf-6f61a212a1cc None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:43.419234 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6babae-620f-4f59-b6f8-0701f3773b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:43.419870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6babae-620f-4f59-b6f8-0701f3773b73 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:43.420188 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6babae-620f-4f59-b6f8-0701f3773b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:43.420481 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6babae-620f-4f59-b6f8-0701f3773b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:43.421125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6babae-620f-4f59-b6f8-0701f3773b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:43.422965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6babae-620f-4f59-b6f8-0701f3773b73 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:43.447883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1046/2131] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:43.448848 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6babae-620f-4f59-b6f8-0701f3773b73 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:43.452320 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ac6babae-620f-4f59-b6f8-0701f3773b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:43.500913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:43.511988 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1086/2132] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:45:43 2022] POST /identity/v3/projects => generated 375 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:43.513322 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fa720-53a9-4211-8f91-6103f4311178 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:43.521622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6babae-620f-4f59-b6f8-0701f3773b73 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:43.581929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fa33d-f4fb-49b0-8d7f-26c92cb95471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:43.583004 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fa33d-f4fb-49b0-8d7f-26c92cb95471 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:43.583347 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fa33d-f4fb-49b0-8d7f-26c92cb95471 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:43.583841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fa33d-f4fb-49b0-8d7f-26c92cb95471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:43.598956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fa33d-f4fb-49b0-8d7f-26c92cb95471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:43.599360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fa720-53a9-4211-8f91-6103f4311178 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:43.602417 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fa33d-f4fb-49b0-8d7f-26c92cb95471 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:43.606313 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff445fb9-c29a-4474-b71c-148295400b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:43.608018 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff445fb9-c29a-4474-b71c-148295400b9e admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:43.608924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff445fb9-c29a-4474-b71c-148295400b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:43.609526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff445fb9-c29a-4474-b71c-148295400b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:43.610145 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff445fb9-c29a-4474-b71c-148295400b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:43.612108 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff445fb9-c29a-4474-b71c-148295400b9e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:43.618757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1047/2133] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:43.619602 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fa33d-f4fb-49b0-8d7f-26c92cb95471 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:43.630577 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fa33d-f4fb-49b0-8d7f-26c92cb95471 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:43.633033 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1087/2134] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:43.640740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf64904-88a6-4654-a046-90e8cefc888f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:43.641489 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf64904-88a6-4654-a046-90e8cefc888f admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:43.641917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf64904-88a6-4654-a046-90e8cefc888f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:43.642297 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf64904-88a6-4654-a046-90e8cefc888f admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:43.642905 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff445fb9-c29a-4474-b71c-148295400b9e admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:43.643107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf64904-88a6-4654-a046-90e8cefc888f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:43.645824 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf64904-88a6-4654-a046-90e8cefc888f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:43.665876 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff445fb9-c29a-4474-b71c-148295400b9e admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:43.674688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc240cf-ed09-490f-9ce9-6513a3176803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:43.675931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc240cf-ed09-490f-9ce9-6513a3176803 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:43.676482 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc240cf-ed09-490f-9ce9-6513a3176803 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:43.676987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc240cf-ed09-490f-9ce9-6513a3176803 admin admin] PATH_INFO: `/v3/projects/0fd092eea8c845d8ad0308ae68f7dc70/users/5baa8dfaea154364a2c6c55fc3b2d9df/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:43.704437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc240cf-ed09-490f-9ce9-6513a3176803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fd092eea8c845d8ad0308ae68f7dc70, user_id=5baa8dfaea154364a2c6c55fc3b2d9df, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:43.708026 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc240cf-ed09-490f-9ce9-6513a3176803 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:43.760346 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1048/2135] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:45:43 2022] POST /identity/v3/users => generated 294 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:43.764729 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf64904-88a6-4654-a046-90e8cefc888f admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:43.778562 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf64904-88a6-4654-a046-90e8cefc888f admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:43.783954 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421c599-7c0b-4480-92de-6ef67ae17faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:43.784902 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421c599-7c0b-4480-92de-6ef67ae17faa admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:43.785179 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421c599-7c0b-4480-92de-6ef67ae17faa admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:43.785449 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421c599-7c0b-4480-92de-6ef67ae17faa admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:43.786167 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421c599-7c0b-4480-92de-6ef67ae17faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:43.787822 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421c599-7c0b-4480-92de-6ef67ae17faa admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:43.800237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1049/2136] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:45:43.800877 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1088/2137] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:43 2022] PUT /identity/v3/projects/0fd092eea8c845d8ad0308ae68f7dc70/users/5baa8dfaea154364a2c6c55fc3b2d9df/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:43.806238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421c599-7c0b-4480-92de-6ef67ae17faa admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:43.813837 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95446401-0983-4fce-9424-a4ece9fd6d06 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:43.814211 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95446401-0983-4fce-9424-a4ece9fd6d06 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:43.814503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95446401-0983-4fce-9424-a4ece9fd6d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:43.817114 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421c599-7c0b-4480-92de-6ef67ae17faa admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:43.861870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-95446401-0983-4fce-9424-a4ece9fd6d06 None None] MFA Rules not processed for user `5baa8dfaea154364a2c6c55fc3b2d9df`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:43.863254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-95446401-0983-4fce-9424-a4ece9fd6d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:43.900586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40d741-9fdc-4cbb-abda-7642c059aa8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:43.901683 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40d741-9fdc-4cbb-abda-7642c059aa8a admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:43.902139 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40d741-9fdc-4cbb-abda-7642c059aa8a admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:43.902721 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40d741-9fdc-4cbb-abda-7642c059aa8a admin admin] PATH_INFO: `/v3/projects/12c6e5cbb15a47398f6338192abc1949/users/e61f970b7a594bec86df1c6bd32259ac/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:43.928541 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40d741-9fdc-4cbb-abda-7642c059aa8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=12c6e5cbb15a47398f6338192abc1949, user_id=e61f970b7a594bec86df1c6bd32259ac, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:43.933263 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40d741-9fdc-4cbb-abda-7642c059aa8a admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:43.983071 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1089/2138] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:43.996912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95446401-0983-4fce-9424-a4ece9fd6d06 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:44.001414 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-95446401-0983-4fce-9424-a4ece9fd6d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:44.009640 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1050/2139] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:43 2022] PUT /identity/v3/projects/12c6e5cbb15a47398f6338192abc1949/users/e61f970b7a594bec86df1c6bd32259ac/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:45:44.011299 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40d741-9fdc-4cbb-abda-7642c059aa8a admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:44.109658 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95446401-0983-4fce-9424-a4ece9fd6d06 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:44.109943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40d741-9fdc-4cbb-abda-7642c059aa8a admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:44.115518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31137ab5-38df-48e7-a974-3d3b7c126cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:44.116163 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31137ab5-38df-48e7-a974-3d3b7c126cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:44.116454 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31137ab5-38df-48e7-a974-3d3b7c126cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:44.117248 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31137ab5-38df-48e7-a974-3d3b7c126cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:44.118073 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31137ab5-38df-48e7-a974-3d3b7c126cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:44.119880 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31137ab5-38df-48e7-a974-3d3b7c126cb5 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:44.136769 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1051/2140] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:44.140938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31137ab5-38df-48e7-a974-3d3b7c126cb5 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:44.151708 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31137ab5-38df-48e7-a974-3d3b7c126cb5 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:44.157446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ff828-81b0-46a4-bdb3-05ec5acabe7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:44.158187 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ff828-81b0-46a4-bdb3-05ec5acabe7f admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:44.158518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ff828-81b0-46a4-bdb3-05ec5acabe7f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:44.160352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ff828-81b0-46a4-bdb3-05ec5acabe7f admin admin] PATH_INFO: `/v3/projects/12c6e5cbb15a47398f6338192abc1949/users/e61f970b7a594bec86df1c6bd32259ac/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:44.160887 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ff828-81b0-46a4-bdb3-05ec5acabe7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=12c6e5cbb15a47398f6338192abc1949, user_id=e61f970b7a594bec86df1c6bd32259ac, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:44.163855 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ff828-81b0-46a4-bdb3-05ec5acabe7f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:44.180059 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1052/2141] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:44 2022] PUT /identity/v3/projects/12c6e5cbb15a47398f6338192abc1949/users/e61f970b7a594bec86df1c6bd32259ac/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:44.181121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600e5d0-dd3d-44d3-902b-907726111327 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:44.181792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600e5d0-dd3d-44d3-902b-907726111327 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:44.182062 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600e5d0-dd3d-44d3-902b-907726111327 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:44.182326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600e5d0-dd3d-44d3-902b-907726111327 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:44.252329 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600e5d0-dd3d-44d3-902b-907726111327 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:44.252675 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600e5d0-dd3d-44d3-902b-907726111327 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:44.262715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1090/2142] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:45:44.265235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ff828-81b0-46a4-bdb3-05ec5acabe7f admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:44.267446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-de4ff828-81b0-46a4-bdb3-05ec5acabe7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:44.283985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600e5d0-dd3d-44d3-902b-907726111327 service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:44.285644 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-9600e5d0-dd3d-44d3-902b-907726111327 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:44.342125 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600e5d0-dd3d-44d3-902b-907726111327 service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:44.373136 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ff828-81b0-46a4-bdb3-05ec5acabe7f admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:44.380920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5660a-46b0-4552-84e5-7d904a3bd90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:44.381758 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5660a-46b0-4552-84e5-7d904a3bd90b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:44.382174 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5660a-46b0-4552-84e5-7d904a3bd90b admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:44.384823 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5660a-46b0-4552-84e5-7d904a3bd90b admin admin] PATH_INFO: `/v3/users/99a53a291a1d4c4d93cbd0a22a01a0b3` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:44.386647 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa5660a-46b0-4552-84e5-7d904a3bd90b admin admin] RBAC: Authorizing `identity:delete_user(user_id=99a53a291a1d4c4d93cbd0a22a01a0b3)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:44.388985 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa5660a-46b0-4552-84e5-7d904a3bd90b admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:44.558679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b4d0c-6038-459c-8960-2aa6f78a7971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:44.560415 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b4d0c-6038-459c-8960-2aa6f78a7971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:44.560923 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b4d0c-6038-459c-8960-2aa6f78a7971 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:44.562438 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b4d0c-6038-459c-8960-2aa6f78a7971 admin admin] PATH_INFO: `/v3/projects/420033d63ac74c6ab0f6ff97d09088f1` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:44.565774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b4d0c-6038-459c-8960-2aa6f78a7971 admin admin] RBAC: Authorizing `identity:delete_project(project_id=420033d63ac74c6ab0f6ff97d09088f1)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:44.568182 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b4d0c-6038-459c-8960-2aa6f78a7971 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:44.661335 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-4aa5660a-46b0-4552-84e5-7d904a3bd90b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99a53a291a1d4c4d93cbd0a22a01a0b3'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:44.678677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-4aa5660a-46b0-4552-84e5-7d904a3bd90b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99a53a291a1d4c4d93cbd0a22a01a0b3'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:44.679988 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-4aa5660a-46b0-4552-84e5-7d904a3bd90b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99a53a291a1d4c4d93cbd0a22a01a0b3'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:44.719488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-4aa5660a-46b0-4552-84e5-7d904a3bd90b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99a53a291a1d4c4d93cbd0a22a01a0b3'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:44.720444 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-4aa5660a-46b0-4552-84e5-7d904a3bd90b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99a53a291a1d4c4d93cbd0a22a01a0b3'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:44.761389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-5e8b4d0c-6038-459c-8960-2aa6f78a7971 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '420033d63ac74c6ab0f6ff97d09088f1', 'request_id': 'req-5e8b4d0c-6038-459c-8960-2aa6f78a7971'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:44.775524 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-5e8b4d0c-6038-459c-8960-2aa6f78a7971 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '420033d63ac74c6ab0f6ff97d09088f1', 'request_id': 'req-5e8b4d0c-6038-459c-8960-2aa6f78a7971'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:44.789772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-5e8b4d0c-6038-459c-8960-2aa6f78a7971 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '420033d63ac74c6ab0f6ff97d09088f1', 'request_id': 'req-5e8b4d0c-6038-459c-8960-2aa6f78a7971'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:44.824379 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1053/2143] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:45:44 2022] DELETE /identity/v3/users/99a53a291a1d4c4d93cbd0a22a01a0b3 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:44.829016 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fefc32-a7a3-4612-8b2b-d14b52b25eac None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:44.830253 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fefc32-a7a3-4612-8b2b-d14b52b25eac None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:44.831457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fefc32-a7a3-4612-8b2b-d14b52b25eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:44.879424 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-99fefc32-a7a3-4612-8b2b-d14b52b25eac None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:44.883519 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-99fefc32-a7a3-4612-8b2b-d14b52b25eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:44.901595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-5e8b4d0c-6038-459c-8960-2aa6f78a7971 admin admin] The token cache is being invalidate because project 420033d63ac74c6ab0f6ff97d09088f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:45:44.904543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-5e8b4d0c-6038-459c-8960-2aa6f78a7971 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:44.904543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-5e8b4d0c-6038-459c-8960-2aa6f78a7971 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:44.905788 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1091/2144] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:45:44 2022] DELETE /identity/v3/projects/420033d63ac74c6ab0f6ff97d09088f1 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:44.911050 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03580538-2771-4239-bdbe-10d12e0ed416 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:44.911337 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03580538-2771-4239-bdbe-10d12e0ed416 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:44.911617 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03580538-2771-4239-bdbe-10d12e0ed416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:44.969990 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-03580538-2771-4239-bdbe-10d12e0ed416 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:44.971226 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-03580538-2771-4239-bdbe-10d12e0ed416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:44.994226 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:45.013622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1054/2145] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:45.026330 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fefc32-a7a3-4612-8b2b-d14b52b25eac None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:45.027867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-99fefc32-a7a3-4612-8b2b-d14b52b25eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:45.121696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1092/2146] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:45.130622 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe694b6-fcc1-4119-a335-4623306884f9 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:45.130910 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe694b6-fcc1-4119-a335-4623306884f9 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:45.131181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe694b6-fcc1-4119-a335-4623306884f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:45.174733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fefc32-a7a3-4612-8b2b-d14b52b25eac None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:45.184411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-bfe694b6-fcc1-4119-a335-4623306884f9 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:45.185698 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-bfe694b6-fcc1-4119-a335-4623306884f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:45.186627 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176c600-0bbc-4436-9f3b-29a7bd648f20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:45.187335 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176c600-0bbc-4436-9f3b-29a7bd648f20 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:45.187612 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176c600-0bbc-4436-9f3b-29a7bd648f20 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:45.187882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176c600-0bbc-4436-9f3b-29a7bd648f20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:45.206395 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176c600-0bbc-4436-9f3b-29a7bd648f20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:45.209686 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176c600-0bbc-4436-9f3b-29a7bd648f20 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:45.210052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1093/2147] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:45.214826 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe694b6-fcc1-4119-a335-4623306884f9 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:45.232109 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe694b6-fcc1-4119-a335-4623306884f9 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:45.235337 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1055/2148] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:45:45.240017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76cb32-51c7-48ff-9a5e-3077559d3590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:45.241223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76cb32-51c7-48ff-9a5e-3077559d3590 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:45.241512 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76cb32-51c7-48ff-9a5e-3077559d3590 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:45.241866 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76cb32-51c7-48ff-9a5e-3077559d3590 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:45.242832 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76cb32-51c7-48ff-9a5e-3077559d3590 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:45:45.243142 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76cb32-51c7-48ff-9a5e-3077559d3590 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:45.245766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76cb32-51c7-48ff-9a5e-3077559d3590 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:45.264745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1094/2149] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:45:45.269538 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176c600-0bbc-4436-9f3b-29a7bd648f20 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:45.282593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176c600-0bbc-4436-9f3b-29a7bd648f20 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:45.310898 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6bec2-4399-4b6d-aec3-f72abbc9394c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:45.311582 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6bec2-4399-4b6d-aec3-f72abbc9394c admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:45.311892 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6bec2-4399-4b6d-aec3-f72abbc9394c admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:45.312181 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6bec2-4399-4b6d-aec3-f72abbc9394c admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:45.338905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b6bec2-4399-4b6d-aec3-f72abbc9394c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:45.342063 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b6bec2-4399-4b6d-aec3-f72abbc9394c admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:45.425212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:45.542724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1056/2150] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:45:45 2022] POST /identity/v3/projects => generated 383 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:45.549477 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76cb32-51c7-48ff-9a5e-3077559d3590 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:45.567274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76cb32-51c7-48ff-9a5e-3077559d3590 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:45.733425 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00863e-3f42-4b34-8b07-c3a184e752f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:45.734145 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00863e-3f42-4b34-8b07-c3a184e752f3 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:45.734513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00863e-3f42-4b34-8b07-c3a184e752f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:45.737376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00863e-3f42-4b34-8b07-c3a184e752f3 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:45.737376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00863e-3f42-4b34-8b07-c3a184e752f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:45.740744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00863e-3f42-4b34-8b07-c3a184e752f3 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:45.866927 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1095/2151] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:45:45 2022] POST /identity/v3/users => generated 292 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:45:45.872565 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6bec2-4399-4b6d-aec3-f72abbc9394c admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:45.895794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6bec2-4399-4b6d-aec3-f72abbc9394c admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:45.925387 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354aee5-9d30-41b5-8da1-d6fe73aca270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:45.926011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354aee5-9d30-41b5-8da1-d6fe73aca270 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:45.926280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354aee5-9d30-41b5-8da1-d6fe73aca270 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:45.927006 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354aee5-9d30-41b5-8da1-d6fe73aca270 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:45.928037 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d354aee5-9d30-41b5-8da1-d6fe73aca270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:45.929461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d354aee5-9d30-41b5-8da1-d6fe73aca270 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:45.955000 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1057/2152] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:45.962770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00863e-3f42-4b34-8b07-c3a184e752f3 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:46.005831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00863e-3f42-4b34-8b07-c3a184e752f3 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:46.044895 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1e9a3-74b5-41c8-96e7-1d4d21fee00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:46.048892 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1e9a3-74b5-41c8-96e7-1d4d21fee00b admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:46.053404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1e9a3-74b5-41c8-96e7-1d4d21fee00b admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:46.055696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1e9a3-74b5-41c8-96e7-1d4d21fee00b admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:46.071759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1e9a3-74b5-41c8-96e7-1d4d21fee00b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:46.097198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1e9a3-74b5-41c8-96e7-1d4d21fee00b admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:46.159125 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1096/2153] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:46.164351 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354aee5-9d30-41b5-8da1-d6fe73aca270 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:46.183845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354aee5-9d30-41b5-8da1-d6fe73aca270 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:46.193001 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cef5f7-b6dc-4761-b147-d9655bc5c7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:46.193928 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cef5f7-b6dc-4761-b147-d9655bc5c7f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:46.195421 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cef5f7-b6dc-4761-b147-d9655bc5c7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:46.196183 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cef5f7-b6dc-4761-b147-d9655bc5c7f9 admin admin] PATH_INFO: `/v3/projects/56b36b89dee9450ca08b3dc87293059c/users/f5bf232018c54c2fae10f8529a55d85d/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:46.231466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cef5f7-b6dc-4761-b147-d9655bc5c7f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b36b89dee9450ca08b3dc87293059c, user_id=f5bf232018c54c2fae10f8529a55d85d, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:46.237538 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cef5f7-b6dc-4761-b147-d9655bc5c7f9 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:46.283331 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1e9a3-74b5-41c8-96e7-1d4d21fee00b admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:46.290140 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-77c1e9a3-74b5-41c8-96e7-1d4d21fee00b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:46.363808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1e9a3-74b5-41c8-96e7-1d4d21fee00b admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:46.371224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1058/2154] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:46 2022] PUT /identity/v3/projects/56b36b89dee9450ca08b3dc87293059c/users/f5bf232018c54c2fae10f8529a55d85d/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:46.399404 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dba647-544b-4351-ab1c-c98896c5fc80 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:46.399773 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dba647-544b-4351-ab1c-c98896c5fc80 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:46.400814 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dba647-544b-4351-ab1c-c98896c5fc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:46.537374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d5754b-c135-4477-8866-248149011130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:46.537939 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5754b-c135-4477-8866-248149011130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:46.539184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5754b-c135-4477-8866-248149011130 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:46.539469 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5754b-c135-4477-8866-248149011130 admin admin] PATH_INFO: `/v3/projects/27cbee4b5c514b0283717305606258dd` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:46.541603 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d5754b-c135-4477-8866-248149011130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27cbee4b5c514b0283717305606258dd)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:46.547961 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d5754b-c135-4477-8866-248149011130 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:46.558477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-77dba647-544b-4351-ab1c-c98896c5fc80 None None] MFA Rules not processed for user `f5bf232018c54c2fae10f8529a55d85d`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:46.560501 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-77dba647-544b-4351-ab1c-c98896c5fc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:46.694752 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:46.719373 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1059/2155] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:46.720624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dba647-544b-4351-ab1c-c98896c5fc80 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:46.748845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dba647-544b-4351-ab1c-c98896c5fc80 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:46.753838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-a6d5754b-c135-4477-8866-248149011130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27cbee4b5c514b0283717305606258dd', 'request_id': 'req-a6d5754b-c135-4477-8866-248149011130'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:46.763185 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-a6d5754b-c135-4477-8866-248149011130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27cbee4b5c514b0283717305606258dd', 'request_id': 'req-a6d5754b-c135-4477-8866-248149011130'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:46.770042 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-a6d5754b-c135-4477-8866-248149011130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27cbee4b5c514b0283717305606258dd', 'request_id': 'req-a6d5754b-c135-4477-8866-248149011130'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:46.827931 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db8df0-665f-4289-92c6-7ae274c54e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:46.828602 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db8df0-665f-4289-92c6-7ae274c54e3e admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:46.828943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db8df0-665f-4289-92c6-7ae274c54e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:46.829328 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db8df0-665f-4289-92c6-7ae274c54e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:46.831023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db8df0-665f-4289-92c6-7ae274c54e3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:46.833904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db8df0-665f-4289-92c6-7ae274c54e3e admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:46.881778 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-a6d5754b-c135-4477-8866-248149011130 admin admin] The token cache is being invalidate because project 27cbee4b5c514b0283717305606258dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:45:46.883017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-a6d5754b-c135-4477-8866-248149011130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:46.883838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-a6d5754b-c135-4477-8866-248149011130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:45:46.886258 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1097/2156] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:45:46 2022] DELETE /identity/v3/projects/27cbee4b5c514b0283717305606258dd => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:46.893452 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d667af-b8e4-4321-bbd0-ebcd87a1d756 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:46.893735 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d667af-b8e4-4321-bbd0-ebcd87a1d756 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:46.893890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d667af-b8e4-4321-bbd0-ebcd87a1d756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:46.940192 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-e7d667af-b8e4-4321-bbd0-ebcd87a1d756 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:46.942376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e7d667af-b8e4-4321-bbd0-ebcd87a1d756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:46.974481 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1060/2157] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:45:46 2022] POST /identity/v3/projects => generated 383 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:46.980203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db8df0-665f-4289-92c6-7ae274c54e3e admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:46.984759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-c6db8df0-665f-4289-92c6-7ae274c54e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:47.118006 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1098/2158] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:45:47.126121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f865f28-0b7a-4cea-97ac-404b0ed0ebdd None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:47.126968 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f865f28-0b7a-4cea-97ac-404b0ed0ebdd None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:47.128576 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f865f28-0b7a-4cea-97ac-404b0ed0ebdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:47.150537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db8df0-665f-4289-92c6-7ae274c54e3e admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:47.180234 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d67e3-170a-445e-bb61-fa0db5021cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:47.191706 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d67e3-170a-445e-bb61-fa0db5021cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:47.192372 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d67e3-170a-445e-bb61-fa0db5021cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:47.192859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d67e3-170a-445e-bb61-fa0db5021cb5 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:47.194129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d67e3-170a-445e-bb61-fa0db5021cb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:47.198410 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d67e3-170a-445e-bb61-fa0db5021cb5 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:47.216294 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-5f865f28-0b7a-4cea-97ac-404b0ed0ebdd None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:47.217499 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-5f865f28-0b7a-4cea-97ac-404b0ed0ebdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:47.249864 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1099/2159] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:47.251729 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f865f28-0b7a-4cea-97ac-404b0ed0ebdd None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:47.267017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f865f28-0b7a-4cea-97ac-404b0ed0ebdd None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:47.272600 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ce14c-8964-4a37-9cae-25057a8bae8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:47.273562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ce14c-8964-4a37-9cae-25057a8bae8e admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:47.273835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ce14c-8964-4a37-9cae-25057a8bae8e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:47.274470 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ce14c-8964-4a37-9cae-25057a8bae8e admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:47.275665 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ce14c-8964-4a37-9cae-25057a8bae8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:45:47.276047 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ce14c-8964-4a37-9cae-25057a8bae8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:47.278280 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ce14c-8964-4a37-9cae-25057a8bae8e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:47.297743 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1100/2160] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 22:45:47.302406 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ce14c-8964-4a37-9cae-25057a8bae8e admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:47.321302 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ce14c-8964-4a37-9cae-25057a8bae8e admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:47.335615 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69037119-de0c-488a-814e-fb3aa9418395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:47.336945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69037119-de0c-488a-814e-fb3aa9418395 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:47.337634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69037119-de0c-488a-814e-fb3aa9418395 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:47.338087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69037119-de0c-488a-814e-fb3aa9418395 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:47.339209 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69037119-de0c-488a-814e-fb3aa9418395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:47.341489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69037119-de0c-488a-814e-fb3aa9418395 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:47.406548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1061/2161] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:45:46 2022] POST /identity/v3/users => generated 292 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:47.411954 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d67e3-170a-445e-bb61-fa0db5021cb5 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:47.433992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d67e3-170a-445e-bb61-fa0db5021cb5 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:47.447737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47196881-2529-44be-95ab-f2ee60e42cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:47.448800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47196881-2529-44be-95ab-f2ee60e42cbb admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:47.449226 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47196881-2529-44be-95ab-f2ee60e42cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:47.449632 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47196881-2529-44be-95ab-f2ee60e42cbb admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:47.450861 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47196881-2529-44be-95ab-f2ee60e42cbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:47.463956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47196881-2529-44be-95ab-f2ee60e42cbb admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:47.495594 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:47.502032 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1062/2162] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:47.503854 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47196881-2529-44be-95ab-f2ee60e42cbb admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:47.522717 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1101/2163] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:45:47 2022] POST /identity/v3/projects => generated 399 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:45:47.525600 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69037119-de0c-488a-814e-fb3aa9418395 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:47.528705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47196881-2529-44be-95ab-f2ee60e42cbb admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:47.789426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21eec5d-ca2c-45af-b2aa-e5a7ae71bb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:47.790512 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eec5d-ca2c-45af-b2aa-e5a7ae71bb7c admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:47.791014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eec5d-ca2c-45af-b2aa-e5a7ae71bb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:47.791515 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eec5d-ca2c-45af-b2aa-e5a7ae71bb7c admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:47.792841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21eec5d-ca2c-45af-b2aa-e5a7ae71bb7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:47.795820 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21eec5d-ca2c-45af-b2aa-e5a7ae71bb7c admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:47.849448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1063/2164] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:47.855786 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21eec5d-ca2c-45af-b2aa-e5a7ae71bb7c admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:47.871843 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21eec5d-ca2c-45af-b2aa-e5a7ae71bb7c admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:47.879766 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a712a4-cc2c-489c-a342-455ccdf59016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:47.880899 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a712a4-cc2c-489c-a342-455ccdf59016 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:47.881062 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a712a4-cc2c-489c-a342-455ccdf59016 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:47.881215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a712a4-cc2c-489c-a342-455ccdf59016 admin admin] PATH_INFO: `/v3/projects/b1a6b315a0bc4dffbc28eb95dd1a5e03/users/70086bc310ce47a192acf05724eef32d/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:47.892951 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69037119-de0c-488a-814e-fb3aa9418395 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:47.905987 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a712a4-cc2c-489c-a342-455ccdf59016 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1a6b315a0bc4dffbc28eb95dd1a5e03, user_id=70086bc310ce47a192acf05724eef32d, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:47.911213 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a712a4-cc2c-489c-a342-455ccdf59016 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:47.922668 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2305c1-65ac-405c-8c7c-039d7f37c1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:47.924771 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2305c1-65ac-405c-8c7c-039d7f37c1fc admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:47.924771 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2305c1-65ac-405c-8c7c-039d7f37c1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:47.924771 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2305c1-65ac-405c-8c7c-039d7f37c1fc admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:47.925685 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2305c1-65ac-405c-8c7c-039d7f37c1fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:47.941586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2305c1-65ac-405c-8c7c-039d7f37c1fc admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:48.028995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1064/2165] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:47 2022] PUT /identity/v3/projects/b1a6b315a0bc4dffbc28eb95dd1a5e03/users/70086bc310ce47a192acf05724eef32d/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:48.059649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef262952-6255-42fa-8a87-e3e6790e3e40 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:48.060954 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef262952-6255-42fa-8a87-e3e6790e3e40 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:48.061441 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef262952-6255-42fa-8a87-e3e6790e3e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:48.106674 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1102/2166] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:45:47 2022] POST /identity/v3/users => generated 300 bytes in 583 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:48.112987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2305c1-65ac-405c-8c7c-039d7f37c1fc admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:48.135882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-ef262952-6255-42fa-8a87-e3e6790e3e40 None None] MFA Rules not processed for user `70086bc310ce47a192acf05724eef32d`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:48.137249 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ef262952-6255-42fa-8a87-e3e6790e3e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:48.157152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2305c1-65ac-405c-8c7c-039d7f37c1fc admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:48.315085 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1065/2167] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:48.325622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef262952-6255-42fa-8a87-e3e6790e3e40 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:48.387292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393477f-678a-405b-8f4a-3cd0d6289ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:48.388198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393477f-678a-405b-8f4a-3cd0d6289ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:48.388670 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393477f-678a-405b-8f4a-3cd0d6289ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:48.389150 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393477f-678a-405b-8f4a-3cd0d6289ea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:48.390317 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393477f-678a-405b-8f4a-3cd0d6289ea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:48.394460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393477f-678a-405b-8f4a-3cd0d6289ea2 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:48.471519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1103/2168] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:48.475430 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393477f-678a-405b-8f4a-3cd0d6289ea2 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:48.500860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393477f-678a-405b-8f4a-3cd0d6289ea2 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:48.514684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676ae10-4ec0-4e5a-a993-a72bfb27c282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:48.516792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676ae10-4ec0-4e5a-a993-a72bfb27c282 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:48.520237 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676ae10-4ec0-4e5a-a993-a72bfb27c282 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:48.520237 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676ae10-4ec0-4e5a-a993-a72bfb27c282 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:48.521222 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6676ae10-4ec0-4e5a-a993-a72bfb27c282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:48.525311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6676ae10-4ec0-4e5a-a993-a72bfb27c282 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:48.555252 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1104/2169] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:48.556816 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676ae10-4ec0-4e5a-a993-a72bfb27c282 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:48.587320 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676ae10-4ec0-4e5a-a993-a72bfb27c282 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:48.587602 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef262952-6255-42fa-8a87-e3e6790e3e40 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:48.594784 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8790b69-ff7f-4ccb-9045-f0f23ed5688f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:48.595650 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8790b69-ff7f-4ccb-9045-f0f23ed5688f admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:48.596036 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8790b69-ff7f-4ccb-9045-f0f23ed5688f admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:48.596816 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8790b69-ff7f-4ccb-9045-f0f23ed5688f admin admin] PATH_INFO: `/v3/projects/16aa21ca40ae44089f6fdb770ada2ae8/users/64442586376049c08dd2dfd98ef7c3f6/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:48.597277 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740f694-7feb-465b-994d-dd749204dc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:48.598363 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740f694-7feb-465b-994d-dd749204dc4f admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:48.598852 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740f694-7feb-465b-994d-dd749204dc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:48.599338 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740f694-7feb-465b-994d-dd749204dc4f admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:48.600536 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740f694-7feb-465b-994d-dd749204dc4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:48.605779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740f694-7feb-465b-994d-dd749204dc4f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:48.634905 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8790b69-ff7f-4ccb-9045-f0f23ed5688f admin admin] RBAC: Authorizing `identity:create_grant(project_id=16aa21ca40ae44089f6fdb770ada2ae8, user_id=64442586376049c08dd2dfd98ef7c3f6, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:48.640896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8790b69-ff7f-4ccb-9045-f0f23ed5688f admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:48.654541 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:48.778576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1066/2170] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:45:48 2022] POST /identity/v3/projects => generated 385 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:48.785435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740f694-7feb-465b-994d-dd749204dc4f admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:48.790260 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1105/2171] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:48 2022] PUT /identity/v3/projects/16aa21ca40ae44089f6fdb770ada2ae8/users/64442586376049c08dd2dfd98ef7c3f6/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:48.811512 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecbb93-74d8-4c8c-bf2d-0d01bf64d8a3 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:48.812432 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecbb93-74d8-4c8c-bf2d-0d01bf64d8a3 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:48.813103 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecbb93-74d8-4c8c-bf2d-0d01bf64d8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:48.933341 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-26ecbb93-74d8-4c8c-bf2d-0d01bf64d8a3 None None] MFA Rules not processed for user `64442586376049c08dd2dfd98ef7c3f6`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:48.940941 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-26ecbb93-74d8-4c8c-bf2d-0d01bf64d8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:49.159588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740f694-7feb-465b-994d-dd749204dc4f admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:49.197292 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c21db-880a-4545-a67e-3f59c441f86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:49.207193 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c21db-880a-4545-a67e-3f59c441f86a admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:49.209439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c21db-880a-4545-a67e-3f59c441f86a admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:49.210852 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c21db-880a-4545-a67e-3f59c441f86a admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:49.216004 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c21db-880a-4545-a67e-3f59c441f86a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:49.222024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c21db-880a-4545-a67e-3f59c441f86a admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:49.226878 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1106/2172] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:48 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:45:49.252914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecbb93-74d8-4c8c-bf2d-0d01bf64d8a3 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:49.255573 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-26ecbb93-74d8-4c8c-bf2d-0d01bf64d8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:49.427576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1067/2173] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:45:48 2022] POST /identity/v3/users => generated 299 bytes in 643 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 22:45:49.429659 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c21db-880a-4545-a67e-3f59c441f86a admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:49.481348 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c21db-880a-4545-a67e-3f59c441f86a admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:49.503009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04cb7c-5af0-4a46-8536-755cc02752fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:49.503763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04cb7c-5af0-4a46-8536-755cc02752fc admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:49.505405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04cb7c-5af0-4a46-8536-755cc02752fc admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:49.505831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04cb7c-5af0-4a46-8536-755cc02752fc admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:49.512187 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04cb7c-5af0-4a46-8536-755cc02752fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:49.525868 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04cb7c-5af0-4a46-8536-755cc02752fc admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:49.591955 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1068/2174] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:49.593546 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04cb7c-5af0-4a46-8536-755cc02752fc admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:49.619758 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecbb93-74d8-4c8c-bf2d-0d01bf64d8a3 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:49.626176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c3120-8680-41ee-af3d-2ea6711bbcf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:49.627658 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c3120-8680-41ee-af3d-2ea6711bbcf4 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:49.628551 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04cb7c-5af0-4a46-8536-755cc02752fc admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:49.628749 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c3120-8680-41ee-af3d-2ea6711bbcf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:49.629779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c3120-8680-41ee-af3d-2ea6711bbcf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:49.654998 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b29175-e837-4d03-b5e7-1f861fbf8c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:49.660866 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b29175-e837-4d03-b5e7-1f861fbf8c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:49.660866 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b29175-e837-4d03-b5e7-1f861fbf8c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:49.660866 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b29175-e837-4d03-b5e7-1f861fbf8c2c admin admin] PATH_INFO: `/v3/projects/db5aff46d56f458dbccb7771c6047724/users/6d03b31fe8624b86bf1679e979d5d617/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:49.662589 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c3120-8680-41ee-af3d-2ea6711bbcf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:49.668353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c3120-8680-41ee-af3d-2ea6711bbcf4 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:49.712653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1107/2175] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:49 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:45:49.719677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b29175-e837-4d03-b5e7-1f861fbf8c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=db5aff46d56f458dbccb7771c6047724, user_id=6d03b31fe8624b86bf1679e979d5d617, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:49.724685 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b29175-e837-4d03-b5e7-1f861fbf8c2c admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:49.832066 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:49.854408 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1069/2176] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:49 2022] PUT /identity/v3/projects/db5aff46d56f458dbccb7771c6047724/users/6d03b31fe8624b86bf1679e979d5d617/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:49.860318 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c3120-8680-41ee-af3d-2ea6711bbcf4 service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:49.888864 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c3120-8680-41ee-af3d-2ea6711bbcf4 service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:50.084562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9be8c-67a5-4053-ae58-b9f4306a9a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:50.085592 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9be8c-67a5-4053-ae58-b9f4306a9a19 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:50.086217 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9be8c-67a5-4053-ae58-b9f4306a9a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:50.086681 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9be8c-67a5-4053-ae58-b9f4306a9a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:50.087683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9be8c-67a5-4053-ae58-b9f4306a9a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:50.089560 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9be8c-67a5-4053-ae58-b9f4306a9a19 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:50.115179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1108/2177] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:45:50.121714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b29175-e837-4d03-b5e7-1f861fbf8c2c admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:50.136009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b29175-e837-4d03-b5e7-1f861fbf8c2c admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:50.142090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2f804-e2ff-46cd-a72a-86aa724998b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:50.142930 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2f804-e2ff-46cd-a72a-86aa724998b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:50.158161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2f804-e2ff-46cd-a72a-86aa724998b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:50.158161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2f804-e2ff-46cd-a72a-86aa724998b9 admin admin] PATH_INFO: `/v3/projects/db5aff46d56f458dbccb7771c6047724/users/6d03b31fe8624b86bf1679e979d5d617/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:50.169563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2f804-e2ff-46cd-a72a-86aa724998b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db5aff46d56f458dbccb7771c6047724, user_id=6d03b31fe8624b86bf1679e979d5d617, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:50.176799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2f804-e2ff-46cd-a72a-86aa724998b9 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:50.217564 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1070/2178] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:45:50 2022] PUT /identity/v3/projects/db5aff46d56f458dbccb7771c6047724/users/6d03b31fe8624b86bf1679e979d5d617/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:45:50.233477 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be76dce-b174-4351-9f4d-a36271e9b206 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:50.233962 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be76dce-b174-4351-9f4d-a36271e9b206 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:50.234439 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be76dce-b174-4351-9f4d-a36271e9b206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:50.348241 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-4be76dce-b174-4351-9f4d-a36271e9b206 None None] MFA Rules not processed for user `6d03b31fe8624b86bf1679e979d5d617`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:50.349802 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-4be76dce-b174-4351-9f4d-a36271e9b206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:50.540509 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1109/2179] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:50.628990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2f804-e2ff-46cd-a72a-86aa724998b9 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:50.633061 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-e4e2f804-e2ff-46cd-a72a-86aa724998b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:50.771360 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2f804-e2ff-46cd-a72a-86aa724998b9 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:50.787165 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1193b6e-283c-4d64-91a0-706c754b76b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:50.787729 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1193b6e-283c-4d64-91a0-706c754b76b8 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:50.789010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1193b6e-283c-4d64-91a0-706c754b76b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:50.789457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1193b6e-283c-4d64-91a0-706c754b76b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:50.793756 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-c1193b6e-283c-4d64-91a0-706c754b76b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:50.819903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:50.903119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1193b6e-283c-4d64-91a0-706c754b76b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:50.904509 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1193b6e-283c-4d64-91a0-706c754b76b8 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:50.918011 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1071/2180] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:51.436866 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:52.786009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:54.382522 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be76dce-b174-4351-9f4d-a36271e9b206 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:54.402358 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be76dce-b174-4351-9f4d-a36271e9b206 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:54.409554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac0fbd-4ea4-491f-b33a-6c91f35fc3d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:54.411213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac0fbd-4ea4-491f-b33a-6c91f35fc3d7 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:54.411772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac0fbd-4ea4-491f-b33a-6c91f35fc3d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:54.412198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac0fbd-4ea4-491f-b33a-6c91f35fc3d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:54.542686 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac0fbd-4ea4-491f-b33a-6c91f35fc3d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:54.544902 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac0fbd-4ea4-491f-b33a-6c91f35fc3d7 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:54.563779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1110/2181] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:55.848894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:55.968414 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:45:56.198035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ea557-cd2c-49fd-b8b8-8b062a0ce770 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:56.198338 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ea557-cd2c-49fd-b8b8-8b062a0ce770 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:56.198620 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ea557-cd2c-49fd-b8b8-8b062a0ce770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:56.257986 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-247ea557-cd2c-49fd-b8b8-8b062a0ce770 None None] MFA Rules not processed for user `e61f970b7a594bec86df1c6bd32259ac`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:45:56.259745 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-247ea557-cd2c-49fd-b8b8-8b062a0ce770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:56.349689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1072/2182] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:45:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:45:56.371997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac0fbd-4ea4-491f-b33a-6c91f35fc3d7 service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:56.373586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-26ac0fbd-4ea4-491f-b33a-6c91f35fc3d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:56.458913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac0fbd-4ea4-491f-b33a-6c91f35fc3d7 service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:56.465686 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca21b5-983f-4dec-b58f-6b6e08d9574d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:56.466617 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca21b5-983f-4dec-b58f-6b6e08d9574d service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:56.466733 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca21b5-983f-4dec-b58f-6b6e08d9574d service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:56.466982 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca21b5-983f-4dec-b58f-6b6e08d9574d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:56.469139 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a2ca21b5-983f-4dec-b58f-6b6e08d9574d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:45:56.556601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca21b5-983f-4dec-b58f-6b6e08d9574d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:56.556601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca21b5-983f-4dec-b58f-6b6e08d9574d service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:56.560986 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1111/2183] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:57.661058 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ea557-cd2c-49fd-b8b8-8b062a0ce770 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:45:57.675342 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ea557-cd2c-49fd-b8b8-8b062a0ce770 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:45:57.679172 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e97f2-ca4d-48d6-a2a2-90ae85dca266 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:45:57.680173 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e97f2-ca4d-48d6-a2a2-90ae85dca266 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:45:57.681158 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e97f2-ca4d-48d6-a2a2-90ae85dca266 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:45:57.682045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e97f2-ca4d-48d6-a2a2-90ae85dca266 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:45:57.699401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e97f2-ca4d-48d6-a2a2-90ae85dca266 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:45:57.701915 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e97f2-ca4d-48d6-a2a2-90ae85dca266 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:45:57.723855 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1073/2184] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:45:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:45:58.921180 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:46:01.207865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca21b5-983f-4dec-b58f-6b6e08d9574d service nova] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:01.281542 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca21b5-983f-4dec-b58f-6b6e08d9574d service nova] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:01.286396 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2d1f7-eb25-413f-8599-27a4c2600148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:01.287034 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2d1f7-eb25-413f-8599-27a4c2600148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:01.287312 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2d1f7-eb25-413f-8599-27a4c2600148 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:01.287591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2d1f7-eb25-413f-8599-27a4c2600148 admin admin] PATH_INFO: `/v3/users/64442586376049c08dd2dfd98ef7c3f6` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:01.289347 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2d1f7-eb25-413f-8599-27a4c2600148 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64442586376049c08dd2dfd98ef7c3f6)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:01.292008 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2d1f7-eb25-413f-8599-27a4c2600148 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:01.637654 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:46:01.656161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-15c2d1f7-eb25-413f-8599-27a4c2600148 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64442586376049c08dd2dfd98ef7c3f6'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:01.656895 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-15c2d1f7-eb25-413f-8599-27a4c2600148 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64442586376049c08dd2dfd98ef7c3f6'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:01.667868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-15c2d1f7-eb25-413f-8599-27a4c2600148 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64442586376049c08dd2dfd98ef7c3f6'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:01.714131 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e97f2-ca4d-48d6-a2a2-90ae85dca266 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:01.715672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-298e97f2-ca4d-48d6-a2a2-90ae85dca266 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:01.730815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e97f2-ca4d-48d6-a2a2-90ae85dca266 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:01.737429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef001cb-9aae-4657-a906-47a81cb7bda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:01.737429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef001cb-9aae-4657-a906-47a81cb7bda4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:01.737429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef001cb-9aae-4657-a906-47a81cb7bda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:01.737429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef001cb-9aae-4657-a906-47a81cb7bda4 admin admin] PATH_INFO: `/v3/users/5baa8dfaea154364a2c6c55fc3b2d9df` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:01.737991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef001cb-9aae-4657-a906-47a81cb7bda4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5baa8dfaea154364a2c6c55fc3b2d9df)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:01.739836 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef001cb-9aae-4657-a906-47a81cb7bda4 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:01.875213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-15c2d1f7-eb25-413f-8599-27a4c2600148 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64442586376049c08dd2dfd98ef7c3f6'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:01.875904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-15c2d1f7-eb25-413f-8599-27a4c2600148 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64442586376049c08dd2dfd98ef7c3f6'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:01.894608 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1112/2185] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:46:01 2022] DELETE /identity/v3/users/64442586376049c08dd2dfd98ef7c3f6 => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:46:01.899946 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef5910-bfc5-4f07-91b8-33fbed333ba3 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:01.900265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef5910-bfc5-4f07-91b8-33fbed333ba3 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:01.900567 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef5910-bfc5-4f07-91b8-33fbed333ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:01.923353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-1eef5910-bfc5-4f07-91b8-33fbed333ba3 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:01.924259 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-1eef5910-bfc5-4f07-91b8-33fbed333ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:01.991907 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1113/2186] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:46:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:46:02.001905 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef5910-bfc5-4f07-91b8-33fbed333ba3 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:02.003474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-1eef5910-bfc5-4f07-91b8-33fbed333ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:02.057061 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:46:02.073268 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-4ef001cb-9aae-4657-a906-47a81cb7bda4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5baa8dfaea154364a2c6c55fc3b2d9df'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:02.074404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef5910-bfc5-4f07-91b8-33fbed333ba3 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:02.078552 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67a0a4-990f-47dc-8976-56d6c22e215c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:02.079474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67a0a4-990f-47dc-8976-56d6c22e215c service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:02.079830 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67a0a4-990f-47dc-8976-56d6c22e215c service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:02.081198 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-4ef001cb-9aae-4657-a906-47a81cb7bda4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5baa8dfaea154364a2c6c55fc3b2d9df'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:02.081912 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-4ef001cb-9aae-4657-a906-47a81cb7bda4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5baa8dfaea154364a2c6c55fc3b2d9df'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:02.083007 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67a0a4-990f-47dc-8976-56d6c22e215c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:02.084856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6a67a0a4-990f-47dc-8976-56d6c22e215c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:02.097647 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-4ef001cb-9aae-4657-a906-47a81cb7bda4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5baa8dfaea154364a2c6c55fc3b2d9df'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:02.098670 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-4ef001cb-9aae-4657-a906-47a81cb7bda4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5baa8dfaea154364a2c6c55fc3b2d9df'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:02.106508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67a0a4-990f-47dc-8976-56d6c22e215c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:02.108598 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67a0a4-990f-47dc-8976-56d6c22e215c service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:02.126066 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1114/2187] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:46:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:46:02.191421 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1074/2188] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:46:01 2022] DELETE /identity/v3/users/5baa8dfaea154364a2c6c55fc3b2d9df => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:46:02.193225 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db6cc8-3648-4937-a1c3-12a281b0199d None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:02.193531 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db6cc8-3648-4937-a1c3-12a281b0199d None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:02.193824 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db6cc8-3648-4937-a1c3-12a281b0199d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:02.219946 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-25db6cc8-3648-4937-a1c3-12a281b0199d None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:02.221254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-25db6cc8-3648-4937-a1c3-12a281b0199d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:02.296280 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1115/2189] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:46:02.307200 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef001cb-9aae-4657-a906-47a81cb7bda4 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:02.308953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-4ef001cb-9aae-4657-a906-47a81cb7bda4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:02.385111 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef001cb-9aae-4657-a906-47a81cb7bda4 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:02.390727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6359eca-affc-4ac7-9fa4-ddce43120692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:02.391592 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6359eca-affc-4ac7-9fa4-ddce43120692 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:02.392093 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6359eca-affc-4ac7-9fa4-ddce43120692 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:02.392414 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6359eca-affc-4ac7-9fa4-ddce43120692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:02.404222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6359eca-affc-4ac7-9fa4-ddce43120692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:02.405662 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6359eca-affc-4ac7-9fa4-ddce43120692 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:02.420803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1075/2190] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:46:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:46:02.806997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db6cc8-3648-4937-a1c3-12a281b0199d None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:02.808604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-25db6cc8-3648-4937-a1c3-12a281b0199d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:02.822137 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db6cc8-3648-4937-a1c3-12a281b0199d None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:02.827301 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb14ba-234b-4bc6-a1dd-18e1e00d5d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:02.827941 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb14ba-234b-4bc6-a1dd-18e1e00d5d9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:02.828244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb14ba-234b-4bc6-a1dd-18e1e00d5d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:02.828519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb14ba-234b-4bc6-a1dd-18e1e00d5d9f admin admin] PATH_INFO: `/v3/projects/16aa21ca40ae44089f6fdb770ada2ae8` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:02.829750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb14ba-234b-4bc6-a1dd-18e1e00d5d9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=16aa21ca40ae44089f6fdb770ada2ae8)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:02.832800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb14ba-234b-4bc6-a1dd-18e1e00d5d9f admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:03.056627 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6359eca-affc-4ac7-9fa4-ddce43120692 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:03.058227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-c6359eca-affc-4ac7-9fa4-ddce43120692 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:03.072543 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6359eca-affc-4ac7-9fa4-ddce43120692 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:03.073779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-e9eb14ba-234b-4bc6-a1dd-18e1e00d5d9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16aa21ca40ae44089f6fdb770ada2ae8', 'request_id': 'req-e9eb14ba-234b-4bc6-a1dd-18e1e00d5d9f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:03.082648 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-e9eb14ba-234b-4bc6-a1dd-18e1e00d5d9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16aa21ca40ae44089f6fdb770ada2ae8', 'request_id': 'req-e9eb14ba-234b-4bc6-a1dd-18e1e00d5d9f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:03.092341 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-e9eb14ba-234b-4bc6-a1dd-18e1e00d5d9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16aa21ca40ae44089f6fdb770ada2ae8', 'request_id': 'req-e9eb14ba-234b-4bc6-a1dd-18e1e00d5d9f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:03.147054 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d835b-ffe9-44ab-9fa7-b3a148e7696e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:03.148200 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d835b-ffe9-44ab-9fa7-b3a148e7696e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:03.148506 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d835b-ffe9-44ab-9fa7-b3a148e7696e admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:03.148797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d835b-ffe9-44ab-9fa7-b3a148e7696e admin admin] PATH_INFO: `/v3/projects/0fd092eea8c845d8ad0308ae68f7dc70` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:03.149956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d835b-ffe9-44ab-9fa7-b3a148e7696e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fd092eea8c845d8ad0308ae68f7dc70)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:03.151894 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d835b-ffe9-44ab-9fa7-b3a148e7696e admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:03.180740 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-e9eb14ba-234b-4bc6-a1dd-18e1e00d5d9f admin admin] The token cache is being invalidate because project 16aa21ca40ae44089f6fdb770ada2ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:46:03.181153 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-e9eb14ba-234b-4bc6-a1dd-18e1e00d5d9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:03.181881 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-e9eb14ba-234b-4bc6-a1dd-18e1e00d5d9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:03.184106 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1116/2191] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:46:02 2022] DELETE /identity/v3/projects/16aa21ca40ae44089f6fdb770ada2ae8 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:46:03.271155 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb14ba-234b-4bc6-a1dd-18e1e00d5d9f admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:03.276094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e9eb14ba-234b-4bc6-a1dd-18e1e00d5d9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:03.290706 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb14ba-234b-4bc6-a1dd-18e1e00d5d9f admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:03.297808 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-9b1d835b-ffe9-44ab-9fa7-b3a148e7696e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fd092eea8c845d8ad0308ae68f7dc70', 'request_id': 'req-9b1d835b-ffe9-44ab-9fa7-b3a148e7696e'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:03.371023 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380a706-c71a-4777-a95b-eb5091825fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:03.371949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380a706-c71a-4777-a95b-eb5091825fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:03.372462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380a706-c71a-4777-a95b-eb5091825fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:03.372752 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380a706-c71a-4777-a95b-eb5091825fa9 admin admin] PATH_INFO: `/v3/users/f5bf232018c54c2fae10f8529a55d85d` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:03.374071 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380a706-c71a-4777-a95b-eb5091825fa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5bf232018c54c2fae10f8529a55d85d)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:03.376070 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380a706-c71a-4777-a95b-eb5091825fa9 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:03.415190 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-9b1d835b-ffe9-44ab-9fa7-b3a148e7696e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fd092eea8c845d8ad0308ae68f7dc70', 'request_id': 'req-9b1d835b-ffe9-44ab-9fa7-b3a148e7696e'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:03.422904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-9b1d835b-ffe9-44ab-9fa7-b3a148e7696e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fd092eea8c845d8ad0308ae68f7dc70', 'request_id': 'req-9b1d835b-ffe9-44ab-9fa7-b3a148e7696e'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:03.434568 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-9b1d835b-ffe9-44ab-9fa7-b3a148e7696e admin admin] The token cache is being invalidate because project 0fd092eea8c845d8ad0308ae68f7dc70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:46:03.434861 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-9b1d835b-ffe9-44ab-9fa7-b3a148e7696e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:03.435706 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-9b1d835b-ffe9-44ab-9fa7-b3a148e7696e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:03.437236 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1076/2192] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:46:03 2022] DELETE /identity/v3/projects/0fd092eea8c845d8ad0308ae68f7dc70 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:46:03.442223 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6795f-ec10-4ceb-9604-87d9e6027f9c None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:03.442593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6795f-ec10-4ceb-9604-87d9e6027f9c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:03.442869 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6795f-ec10-4ceb-9604-87d9e6027f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:03.468916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-19a6795f-ec10-4ceb-9604-87d9e6027f9c None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:03.470130 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-19a6795f-ec10-4ceb-9604-87d9e6027f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:03.479251 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1077/2193] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:46:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:46:03.493252 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a999a03-349b-49a8-b967-9e8b54e2addf None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:03.493744 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a999a03-349b-49a8-b967-9e8b54e2addf None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:03.494044 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a999a03-349b-49a8-b967-9e8b54e2addf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:03.514362 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-0a999a03-349b-49a8-b967-9e8b54e2addf None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:03.515570 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-0a999a03-349b-49a8-b967-9e8b54e2addf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:03.524307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1078/2194] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:46:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:46:03.528571 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a999a03-349b-49a8-b967-9e8b54e2addf None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:03.538974 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a999a03-349b-49a8-b967-9e8b54e2addf None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:03.543829 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2fa36-f899-4132-af1f-665c36d19761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:03.544459 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2fa36-f899-4132-af1f-665c36d19761 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:03.544739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2fa36-f899-4132-af1f-665c36d19761 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:03.545012 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2fa36-f899-4132-af1f-665c36d19761 admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:03.545657 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2fa36-f899-4132-af1f-665c36d19761 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 22:46:03.545958 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2fa36-f899-4132-af1f-665c36d19761 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:03.547579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2fa36-f899-4132-af1f-665c36d19761 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:03.559508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1079/2195] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 22:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 22:46:03.563731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2fa36-f899-4132-af1f-665c36d19761 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:03.574977 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2fa36-f899-4132-af1f-665c36d19761 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:03.580934 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2c2e9-b99e-4dd4-8067-29717ace49fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:03.581582 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2c2e9-b99e-4dd4-8067-29717ace49fb admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:03.581875 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2c2e9-b99e-4dd4-8067-29717ace49fb admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:03.582128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2c2e9-b99e-4dd4-8067-29717ace49fb admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:03.582810 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2c2e9-b99e-4dd4-8067-29717ace49fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:03.585306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2c2e9-b99e-4dd4-8067-29717ace49fb admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:03.696407 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8380a706-c71a-4777-a95b-eb5091825fa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5bf232018c54c2fae10f8529a55d85d'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:03.697080 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8380a706-c71a-4777-a95b-eb5091825fa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5bf232018c54c2fae10f8529a55d85d'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:03.704744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8380a706-c71a-4777-a95b-eb5091825fa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5bf232018c54c2fae10f8529a55d85d'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:03.735960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1080/2196] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 22:46:03 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:46:03.740054 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2c2e9-b99e-4dd4-8067-29717ace49fb admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:03.741777 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-2ec2c2e9-b99e-4dd4-8067-29717ace49fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:03.802077 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8380a706-c71a-4777-a95b-eb5091825fa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5bf232018c54c2fae10f8529a55d85d'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:03.802884 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8380a706-c71a-4777-a95b-eb5091825fa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5bf232018c54c2fae10f8529a55d85d'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:03.819852 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2c2e9-b99e-4dd4-8067-29717ace49fb admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:03.824732 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110c8f7-dba4-45ee-ab3e-52ebb4a5a3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:03.824896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1117/2197] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:46:03 2022] DELETE /identity/v3/users/f5bf232018c54c2fae10f8529a55d85d => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:46:03.825342 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110c8f7-dba4-45ee-ab3e-52ebb4a5a3c5 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:03.825614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110c8f7-dba4-45ee-ab3e-52ebb4a5a3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:03.826023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110c8f7-dba4-45ee-ab3e-52ebb4a5a3c5 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:03.826658 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3110c8f7-dba4-45ee-ab3e-52ebb4a5a3c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:03.828608 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3110c8f7-dba4-45ee-ab3e-52ebb4a5a3c5 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:03.829766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793eb47b-87a5-4c3f-9644-c87c63105c6b None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:03.830393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793eb47b-87a5-4c3f-9644-c87c63105c6b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:03.830743 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793eb47b-87a5-4c3f-9644-c87c63105c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:03.850770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-793eb47b-87a5-4c3f-9644-c87c63105c6b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:03.852020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-793eb47b-87a5-4c3f-9644-c87c63105c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:03.893681 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:46:03.914547 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1118/2198] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:46:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:46:03.929620 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793eb47b-87a5-4c3f-9644-c87c63105c6b None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:03.931223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-793eb47b-87a5-4c3f-9644-c87c63105c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:03.945953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1081/2199] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 22:46:03 2022] POST /identity/v3/users => generated 291 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 22:46:03.952063 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110c8f7-dba4-45ee-ab3e-52ebb4a5a3c5 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:03.963244 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110c8f7-dba4-45ee-ab3e-52ebb4a5a3c5 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:03.969258 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf271ae-848f-4b86-a3a2-1e2f2b2f1385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:03.969992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf271ae-848f-4b86-a3a2-1e2f2b2f1385 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:03.970268 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf271ae-848f-4b86-a3a2-1e2f2b2f1385 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:03.970536 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf271ae-848f-4b86-a3a2-1e2f2b2f1385 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:03.971171 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf271ae-848f-4b86-a3a2-1e2f2b2f1385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:03.972894 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf271ae-848f-4b86-a3a2-1e2f2b2f1385 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:03.989486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1082/2200] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 22:46:03.992115 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf271ae-848f-4b86-a3a2-1e2f2b2f1385 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:04.003034 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf271ae-848f-4b86-a3a2-1e2f2b2f1385 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:04.006197 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793eb47b-87a5-4c3f-9644-c87c63105c6b None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:04.009657 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2807b62-1a78-4a85-a18b-51f396096592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:04.010385 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2807b62-1a78-4a85-a18b-51f396096592 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:04.010775 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2807b62-1a78-4a85-a18b-51f396096592 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:04.011405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2807b62-1a78-4a85-a18b-51f396096592 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:04.011949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2807b62-1a78-4a85-a18b-51f396096592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:04.012615 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd8462-5bf0-41c0-8d90-7d2aef587318 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:04.013770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd8462-5bf0-41c0-8d90-7d2aef587318 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:04.014295 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd8462-5bf0-41c0-8d90-7d2aef587318 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:04.014650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2807b62-1a78-4a85-a18b-51f396096592 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:04.014898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd8462-5bf0-41c0-8d90-7d2aef587318 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:04.030701 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1083/2201] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 22:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 22:46:04.030917 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd8462-5bf0-41c0-8d90-7d2aef587318 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:04.032093 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd8462-5bf0-41c0-8d90-7d2aef587318 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:04.034725 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2807b62-1a78-4a85-a18b-51f396096592 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:04.045511 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2807b62-1a78-4a85-a18b-51f396096592 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:04.046802 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1119/2202] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:46:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:46:04.051911 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e23381-278f-4ed1-97d7-1e235c818d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:04.053370 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e23381-278f-4ed1-97d7-1e235c818d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:04.053724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e23381-278f-4ed1-97d7-1e235c818d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:04.054661 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e23381-278f-4ed1-97d7-1e235c818d72 admin admin] PATH_INFO: `/v3/projects/2c0dcefe769247eaa49da038e336bc8a/users/484c09f238bd4b0d91b6aeaee913f7d5/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:04.073191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e23381-278f-4ed1-97d7-1e235c818d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c0dcefe769247eaa49da038e336bc8a, user_id=484c09f238bd4b0d91b6aeaee913f7d5, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:04.075874 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e23381-278f-4ed1-97d7-1e235c818d72 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:04.169271 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1084/2203] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 22:46:04 2022] PUT /identity/v3/projects/2c0dcefe769247eaa49da038e336bc8a/users/484c09f238bd4b0d91b6aeaee913f7d5/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:46:04.174374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91534f33-f145-4841-8a9a-1a65f17d01c2 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:04.174668 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91534f33-f145-4841-8a9a-1a65f17d01c2 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:04.174975 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91534f33-f145-4841-8a9a-1a65f17d01c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:04.186116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:46:04.196728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-91534f33-f145-4841-8a9a-1a65f17d01c2 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:04.197963 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-91534f33-f145-4841-8a9a-1a65f17d01c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:04.258705 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1120/2204] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:46:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:46:04.275208 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e23381-278f-4ed1-97d7-1e235c818d72 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:04.282019 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b1e23381-278f-4ed1-97d7-1e235c818d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:04.357015 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e23381-278f-4ed1-97d7-1e235c818d72 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:04.361061 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4a4a5-3e0b-4949-99a1-05ed78a00e07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:04.367098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4a4a5-3e0b-4949-99a1-05ed78a00e07 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:04.367098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4a4a5-3e0b-4949-99a1-05ed78a00e07 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:04.367098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4a4a5-3e0b-4949-99a1-05ed78a00e07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:04.374091 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4a4a5-3e0b-4949-99a1-05ed78a00e07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:04.376100 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4a4a5-3e0b-4949-99a1-05ed78a00e07 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:04.391619 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1085/2205] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:46:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:46:04.541593 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91534f33-f145-4841-8a9a-1a65f17d01c2 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:04.543032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-91534f33-f145-4841-8a9a-1a65f17d01c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:04.558522 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91534f33-f145-4841-8a9a-1a65f17d01c2 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:04.566946 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00496518-da30-45a7-a993-8c5ef3f6da06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:04.567859 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00496518-da30-45a7-a993-8c5ef3f6da06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:04.568353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00496518-da30-45a7-a993-8c5ef3f6da06 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:04.568841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00496518-da30-45a7-a993-8c5ef3f6da06 admin admin] PATH_INFO: `/v3/projects/56b36b89dee9450ca08b3dc87293059c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:04.570569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00496518-da30-45a7-a993-8c5ef3f6da06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56b36b89dee9450ca08b3dc87293059c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:04.572750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00496518-da30-45a7-a993-8c5ef3f6da06 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:04.877493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-00496518-da30-45a7-a993-8c5ef3f6da06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56b36b89dee9450ca08b3dc87293059c', 'request_id': 'req-00496518-da30-45a7-a993-8c5ef3f6da06'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:04.885655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-00496518-da30-45a7-a993-8c5ef3f6da06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56b36b89dee9450ca08b3dc87293059c', 'request_id': 'req-00496518-da30-45a7-a993-8c5ef3f6da06'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:04.892935 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-00496518-da30-45a7-a993-8c5ef3f6da06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56b36b89dee9450ca08b3dc87293059c', 'request_id': 'req-00496518-da30-45a7-a993-8c5ef3f6da06'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:04.980529 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-00496518-da30-45a7-a993-8c5ef3f6da06 admin admin] The token cache is being invalidate because project 56b36b89dee9450ca08b3dc87293059c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:46:04.981423 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-00496518-da30-45a7-a993-8c5ef3f6da06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:04.984061 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-00496518-da30-45a7-a993-8c5ef3f6da06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:04.985824 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1121/2206] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:46:04 2022] DELETE /identity/v3/projects/56b36b89dee9450ca08b3dc87293059c => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:46:04.989978 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4a4a5-3e0b-4949-99a1-05ed78a00e07 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:05.000366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4a4a5-3e0b-4949-99a1-05ed78a00e07 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:05.063237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2ab8d-0247-484c-86cd-39e7d3c12e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:05.063877 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2ab8d-0247-484c-86cd-39e7d3c12e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:05.064176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2ab8d-0247-484c-86cd-39e7d3c12e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:05.064511 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2ab8d-0247-484c-86cd-39e7d3c12e47 admin admin] PATH_INFO: `/v3/users/70086bc310ce47a192acf05724eef32d` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:05.065633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2ab8d-0247-484c-86cd-39e7d3c12e47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70086bc310ce47a192acf05724eef32d)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:05.067599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2ab8d-0247-484c-86cd-39e7d3c12e47 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:05.257334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1cc2ab8d-0247-484c-86cd-39e7d3c12e47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70086bc310ce47a192acf05724eef32d'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:05.264887 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1cc2ab8d-0247-484c-86cd-39e7d3c12e47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70086bc310ce47a192acf05724eef32d'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:05.265633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1cc2ab8d-0247-484c-86cd-39e7d3c12e47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70086bc310ce47a192acf05724eef32d'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:05.281995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1cc2ab8d-0247-484c-86cd-39e7d3c12e47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70086bc310ce47a192acf05724eef32d'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:05.282753 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1cc2ab8d-0247-484c-86cd-39e7d3c12e47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70086bc310ce47a192acf05724eef32d'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:05.391823 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1086/2207] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:46:04 2022] DELETE /identity/v3/users/70086bc310ce47a192acf05724eef32d => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:46:06.008351 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00496518-da30-45a7-a993-8c5ef3f6da06 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:06.011043 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-00496518-da30-45a7-a993-8c5ef3f6da06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:06.030502 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00496518-da30-45a7-a993-8c5ef3f6da06 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:06.108642 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3364d-97de-4730-9bed-a3adb41a4372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:06.109901 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3364d-97de-4730-9bed-a3adb41a4372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:06.109901 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3364d-97de-4730-9bed-a3adb41a4372 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:06.110133 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3364d-97de-4730-9bed-a3adb41a4372 admin admin] PATH_INFO: `/v3/projects/b1a6b315a0bc4dffbc28eb95dd1a5e03` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:06.111681 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3364d-97de-4730-9bed-a3adb41a4372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1a6b315a0bc4dffbc28eb95dd1a5e03)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:06.113805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3364d-97de-4730-9bed-a3adb41a4372 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:06.141009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e8c2e-b5ef-4a0e-ad70-a59f74c3d44f None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:06.141009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e8c2e-b5ef-4a0e-ad70-a59f74c3d44f None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:06.141009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e8c2e-b5ef-4a0e-ad70-a59f74c3d44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:06.207285 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-f23e8c2e-b5ef-4a0e-ad70-a59f74c3d44f None None] MFA Rules not processed for user `484c09f238bd4b0d91b6aeaee913f7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:06.209001 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f23e8c2e-b5ef-4a0e-ad70-a59f74c3d44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:06.261012 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-2cc3364d-97de-4730-9bed-a3adb41a4372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1a6b315a0bc4dffbc28eb95dd1a5e03', 'request_id': 'req-2cc3364d-97de-4730-9bed-a3adb41a4372'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:06.268804 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-2cc3364d-97de-4730-9bed-a3adb41a4372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1a6b315a0bc4dffbc28eb95dd1a5e03', 'request_id': 'req-2cc3364d-97de-4730-9bed-a3adb41a4372'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:06.283976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-2cc3364d-97de-4730-9bed-a3adb41a4372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1a6b315a0bc4dffbc28eb95dd1a5e03', 'request_id': 'req-2cc3364d-97de-4730-9bed-a3adb41a4372'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:06.329699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1087/2208] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:46:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:46:06.342531 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e8c2e-b5ef-4a0e-ad70-a59f74c3d44f None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:06.345160 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f23e8c2e-b5ef-4a0e-ad70-a59f74c3d44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:06.386665 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-2cc3364d-97de-4730-9bed-a3adb41a4372 admin admin] The token cache is being invalidate because project b1a6b315a0bc4dffbc28eb95dd1a5e03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:46:06.386959 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-2cc3364d-97de-4730-9bed-a3adb41a4372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:06.387622 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-2cc3364d-97de-4730-9bed-a3adb41a4372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:06.389313 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1122/2209] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:46:06 2022] DELETE /identity/v3/projects/b1a6b315a0bc4dffbc28eb95dd1a5e03 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:46:06.394324 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3364d-97de-4730-9bed-a3adb41a4372 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:06.395739 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-2cc3364d-97de-4730-9bed-a3adb41a4372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:06.416921 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e8c2e-b5ef-4a0e-ad70-a59f74c3d44f None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:06.481259 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3364d-97de-4730-9bed-a3adb41a4372 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:06.485752 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae2bb8-d780-41a6-a89e-4fc9f0b946dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:06.486679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae2bb8-d780-41a6-a89e-4fc9f0b946dc service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:06.487026 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1eb0a-4f6a-4bb5-894d-b94ccdb0de8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:06.487163 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae2bb8-d780-41a6-a89e-4fc9f0b946dc service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:06.487433 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae2bb8-d780-41a6-a89e-4fc9f0b946dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:06.487904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1eb0a-4f6a-4bb5-894d-b94ccdb0de8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:06.488270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1eb0a-4f6a-4bb5-894d-b94ccdb0de8f admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:06.488664 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1eb0a-4f6a-4bb5-894d-b94ccdb0de8f admin admin] PATH_INFO: `/v3/users/6d03b31fe8624b86bf1679e979d5d617` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:06.489853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-e5ae2bb8-d780-41a6-a89e-4fc9f0b946dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:06.489980 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc1eb0a-4f6a-4bb5-894d-b94ccdb0de8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d03b31fe8624b86bf1679e979d5d617)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:06.492594 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc1eb0a-4f6a-4bb5-894d-b94ccdb0de8f admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:06.593646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae2bb8-d780-41a6-a89e-4fc9f0b946dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:06.595174 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae2bb8-d780-41a6-a89e-4fc9f0b946dc service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:06.606305 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1088/2210] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:46:06 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:46:06.825279 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ebc1eb0a-4f6a-4bb5-894d-b94ccdb0de8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d03b31fe8624b86bf1679e979d5d617'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:06.826082 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ebc1eb0a-4f6a-4bb5-894d-b94ccdb0de8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d03b31fe8624b86bf1679e979d5d617'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:06.833522 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ebc1eb0a-4f6a-4bb5-894d-b94ccdb0de8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d03b31fe8624b86bf1679e979d5d617'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:06.912547 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ebc1eb0a-4f6a-4bb5-894d-b94ccdb0de8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d03b31fe8624b86bf1679e979d5d617'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:06.913741 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ebc1eb0a-4f6a-4bb5-894d-b94ccdb0de8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d03b31fe8624b86bf1679e979d5d617'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:06.939284 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1123/2211] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 22:46:06 2022] DELETE /identity/v3/users/6d03b31fe8624b86bf1679e979d5d617 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 22:46:07.026378 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:46:07.268187 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:46:07.460478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae2bb8-d780-41a6-a89e-4fc9f0b946dc service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:07.472310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae2bb8-d780-41a6-a89e-4fc9f0b946dc service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:07.552871 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb95f9-60b4-4365-a261-49b2368b19e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:07.553715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb95f9-60b4-4365-a261-49b2368b19e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:07.554133 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb95f9-60b4-4365-a261-49b2368b19e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:07.554465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb95f9-60b4-4365-a261-49b2368b19e0 admin admin] PATH_INFO: `/v3/projects/db5aff46d56f458dbccb7771c6047724` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:07.555698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb95f9-60b4-4365-a261-49b2368b19e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db5aff46d56f458dbccb7771c6047724)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:07.557654 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb95f9-60b4-4365-a261-49b2368b19e0 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:07.702477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b0eb95f9-60b4-4365-a261-49b2368b19e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db5aff46d56f458dbccb7771c6047724', 'request_id': 'req-b0eb95f9-60b4-4365-a261-49b2368b19e0'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:07.813699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b0eb95f9-60b4-4365-a261-49b2368b19e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db5aff46d56f458dbccb7771c6047724', 'request_id': 'req-b0eb95f9-60b4-4365-a261-49b2368b19e0'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:07.821312 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b0eb95f9-60b4-4365-a261-49b2368b19e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db5aff46d56f458dbccb7771c6047724', 'request_id': 'req-b0eb95f9-60b4-4365-a261-49b2368b19e0'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:07.834645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b0eb95f9-60b4-4365-a261-49b2368b19e0 admin admin] The token cache is being invalidate because project db5aff46d56f458dbccb7771c6047724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 22:46:07.835579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b0eb95f9-60b4-4365-a261-49b2368b19e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:07.836374 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b0eb95f9-60b4-4365-a261-49b2368b19e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 22:46:07.837886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1089/2212] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 22:46:07 2022] DELETE /identity/v3/projects/db5aff46d56f458dbccb7771c6047724 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 22:46:08.576438 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf68c1-539e-4d9a-8e4d-8039220983a5 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:08.576974 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf68c1-539e-4d9a-8e4d-8039220983a5 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:08.576974 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf68c1-539e-4d9a-8e4d-8039220983a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:08.600329 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-d8cf68c1-539e-4d9a-8e4d-8039220983a5 None None] MFA Rules not processed for user `484c09f238bd4b0d91b6aeaee913f7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:08.602836 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-d8cf68c1-539e-4d9a-8e4d-8039220983a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:08.672780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1124/2213] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:46:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:46:09.448126 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:46:09.717200 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:46:11.019820 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:46:12.025632 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:46:14.571221 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813a92f-a34f-4d45-8c4a-0d69d5b1ec76 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:14.571903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813a92f-a34f-4d45-8c4a-0d69d5b1ec76 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:14.571903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813a92f-a34f-4d45-8c4a-0d69d5b1ec76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:14.592605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-a813a92f-a34f-4d45-8c4a-0d69d5b1ec76 None None] MFA Rules not processed for user `cc2f5c40aedf47dcb6374d64fcb04add`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:14.593870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a813a92f-a34f-4d45-8c4a-0d69d5b1ec76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:14.654790 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1090/2214] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:46:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:46:16.634882 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb6d51-646e-40f9-b4bd-3d19ac776e72 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:16.636304 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb6d51-646e-40f9-b4bd-3d19ac776e72 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:16.636740 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb6d51-646e-40f9-b4bd-3d19ac776e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:16.658023 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-ebfb6d51-646e-40f9-b4bd-3d19ac776e72 None None] MFA Rules not processed for user `980157efae8f43b59dc063fda04c0348`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:16.659241 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-ebfb6d51-646e-40f9-b4bd-3d19ac776e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:16.751027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1125/2215] 10.222.0.59 () {58 vars in 1009 bytes} [Mon Aug 29 22:46:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:46:16.794233 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a813a92f-a34f-4d45-8c4a-0d69d5b1ec76 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:16.796213 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a813a92f-a34f-4d45-8c4a-0d69d5b1ec76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:16.883710 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a813a92f-a34f-4d45-8c4a-0d69d5b1ec76 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:16.889340 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffc203-a50a-483c-aa98-3744e80ad17a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f0503ae1b24ce6a8adb3cfd7973f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:16.890017 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffc203-a50a-483c-aa98-3744e80ad17a service swift] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:16.890312 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffc203-a50a-483c-aa98-3744e80ad17a service swift] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:16.890550 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffc203-a50a-483c-aa98-3744e80ad17a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:16.900050 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ffc203-a50a-483c-aa98-3744e80ad17a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:16.901833 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ffc203-a50a-483c-aa98-3744e80ad17a service swift] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:16.910212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1091/2216] 10.222.0.59 () {60 vars in 1384 bytes} [Mon Aug 29 22:46:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 22:46:21.390921 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26295243-0702-4074-a05c-fcaa2b55a544 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:21.393898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26295243-0702-4074-a05c-fcaa2b55a544 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:21.393898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26295243-0702-4074-a05c-fcaa2b55a544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:21.415781 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-26295243-0702-4074-a05c-fcaa2b55a544 None None] MFA Rules not processed for user `21feac769b4b48508163f535cdedfbe5`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:21.417482 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-26295243-0702-4074-a05c-fcaa2b55a544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:21.434257 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:46:21.490172 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1126/2217] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:46:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:46:26.907732 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:46:31.807473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=16 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:46:40.937690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63428a-96a5-4fdd-9082-b4ab17868a6c None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:40.938315 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63428a-96a5-4fdd-9082-b4ab17868a6c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:40.938419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63428a-96a5-4fdd-9082-b4ab17868a6c None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:40.939812 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1092/2218] 10.222.0.59 () {54 vars in 921 bytes} [Mon Aug 29 22:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 22:46:40.947061 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6179b-5b94-4c8b-9587-85eaefa2ea77 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:40.947769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6179b-5b94-4c8b-9587-85eaefa2ea77 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:40.948068 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6179b-5b94-4c8b-9587-85eaefa2ea77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:40.972406 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-84e6179b-5b94-4c8b-9587-85eaefa2ea77 None None] MFA Rules not processed for user `7f21105679f245a19ea4850e6c010dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:40.973722 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-84e6179b-5b94-4c8b-9587-85eaefa2ea77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:41.084782 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1127/2219] 10.222.0.59 () {58 vars in 1018 bytes} [Mon Aug 29 22:46:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:46:41.097369 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63428a-96a5-4fdd-9082-b4ab17868a6c None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:41.099176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ca63428a-96a5-4fdd-9082-b4ab17868a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:41.160770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63428a-96a5-4fdd-9082-b4ab17868a6c None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:41.164311 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6aac1-0995-4047-a661-d88429f66d4d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4db08aa2e4ef4478a3909a876754814c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:41.165289 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6aac1-0995-4047-a661-d88429f66d4d service glance] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:41.165568 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6aac1-0995-4047-a661-d88429f66d4d service glance] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:41.165842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6aac1-0995-4047-a661-d88429f66d4d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:41.176152 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e6aac1-0995-4047-a661-d88429f66d4d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:41.177575 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e6aac1-0995-4047-a661-d88429f66d4d service glance] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:41.188324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1093/2220] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:46:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:46:41.433103 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51dbad6-f2e7-4d0c-a935-099e5da01283 None None] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:41.433321 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51dbad6-f2e7-4d0c-a935-099e5da01283 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:41.433558 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51dbad6-f2e7-4d0c-a935-099e5da01283 None None] PATH_INFO: `/` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:41.434959 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1128/2221] 10.222.0.59 () {54 vars in 921 bytes} [Mon Aug 29 22:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:46:41.440403 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2301f2-93cd-4f70-9f21-091cde74e50c None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:41.440844 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2301f2-93cd-4f70-9f21-091cde74e50c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:41.441226 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2301f2-93cd-4f70-9f21-091cde74e50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:41.465464 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-ce2301f2-93cd-4f70-9f21-091cde74e50c None None] MFA Rules not processed for user `cb802d01e71b486cb40c6388430d674d`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:41.467090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ce2301f2-93cd-4f70-9f21-091cde74e50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:41.530605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1094/2222] 10.222.0.59 () {58 vars in 1018 bytes} [Mon Aug 29 22:46:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:46:41.543232 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51dbad6-f2e7-4d0c-a935-099e5da01283 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:41.544961 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e51dbad6-f2e7-4d0c-a935-099e5da01283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:41.561840 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51dbad6-f2e7-4d0c-a935-099e5da01283 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:41.565446 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b87edc-433b-477a-931e-64096662b32f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:41.566045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b87edc-433b-477a-931e-64096662b32f service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:41.566323 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b87edc-433b-477a-931e-64096662b32f service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:41.566610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b87edc-433b-477a-931e-64096662b32f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:41.575175 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b87edc-433b-477a-931e-64096662b32f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:41.576649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b87edc-433b-477a-931e-64096662b32f service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:41.585488 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1129/2223] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:46:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:46:41.673111 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbea7fd-43af-4b8d-91b2-52aef5d8aefe None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:41.673654 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbea7fd-43af-4b8d-91b2-52aef5d8aefe None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:41.674025 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbea7fd-43af-4b8d-91b2-52aef5d8aefe None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:41.675469 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1095/2224] 10.222.0.59 () {54 vars in 921 bytes} [Mon Aug 29 22:46:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 22:46:41.682183 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cdadb-d9cd-49a6-bae3-58e62f4b2851 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:41.682465 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cdadb-d9cd-49a6-bae3-58e62f4b2851 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:41.682741 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cdadb-d9cd-49a6-bae3-58e62f4b2851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:41.703174 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-b43cdadb-d9cd-49a6-bae3-58e62f4b2851 None None] MFA Rules not processed for user `7f21105679f245a19ea4850e6c010dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:41.704373 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b43cdadb-d9cd-49a6-bae3-58e62f4b2851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:41.723078 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1130/2225] 10.222.0.59 () {58 vars in 1018 bytes} [Mon Aug 29 22:46:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:46:41.734922 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbea7fd-43af-4b8d-91b2-52aef5d8aefe None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:41.747044 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbea7fd-43af-4b8d-91b2-52aef5d8aefe None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:41.750671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8c489-08f5-4e5e-9982-0101539047e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:41.751303 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8c489-08f5-4e5e-9982-0101539047e9 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:41.751572 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8c489-08f5-4e5e-9982-0101539047e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:41.751843 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8c489-08f5-4e5e-9982-0101539047e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:41.763000 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8c489-08f5-4e5e-9982-0101539047e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:41.764448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8c489-08f5-4e5e-9982-0101539047e9 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:41.775237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1096/2226] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:46:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:46:49.266999 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48507d2b-c6eb-48ae-8197-bc4f6261f632 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:49.267470 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48507d2b-c6eb-48ae-8197-bc4f6261f632 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:49.267570 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48507d2b-c6eb-48ae-8197-bc4f6261f632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:49.288937 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-48507d2b-c6eb-48ae-8197-bc4f6261f632 None None] MFA Rules not processed for user `980157efae8f43b59dc063fda04c0348`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:46:49.290742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-48507d2b-c6eb-48ae-8197-bc4f6261f632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:49.304823 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1131/2227] 10.222.0.59 () {58 vars in 1009 bytes} [Mon Aug 29 22:46:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:46:49.322341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8c489-08f5-4e5e-9982-0101539047e9 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:49.335706 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8c489-08f5-4e5e-9982-0101539047e9 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:49.342024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e836f90-fff1-4798-9301-17fd0f41e51b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f0503ae1b24ce6a8adb3cfd7973f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:49.342624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e836f90-fff1-4798-9301-17fd0f41e51b service swift] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:49.343077 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e836f90-fff1-4798-9301-17fd0f41e51b service swift] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:49.343222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e836f90-fff1-4798-9301-17fd0f41e51b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:49.353157 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e836f90-fff1-4798-9301-17fd0f41e51b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:49.354828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e836f90-fff1-4798-9301-17fd0f41e51b service swift] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:49.375498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1097/2228] 10.222.0.59 () {60 vars in 1384 bytes} [Mon Aug 29 22:46:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 22:46:49.652913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48507d2b-c6eb-48ae-8197-bc4f6261f632 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:46:49.664568 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48507d2b-c6eb-48ae-8197-bc4f6261f632 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:46:49.673850 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d4014-3540-49f5-8d2a-54407d3dc689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:46:49.675057 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d4014-3540-49f5-8d2a-54407d3dc689 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:46:49.675707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d4014-3540-49f5-8d2a-54407d3dc689 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:46:49.676788 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d4014-3540-49f5-8d2a-54407d3dc689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:46:49.679486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-166d4014-3540-49f5-8d2a-54407d3dc689 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:46:49.811445 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d4014-3540-49f5-8d2a-54407d3dc689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:46:49.813953 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d4014-3540-49f5-8d2a-54407d3dc689 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:46:49.832102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1132/2229] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:46:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:46:59.370731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=61 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:46:59.825868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=73 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:47:05.798900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e836f90-fff1-4798-9301-17fd0f41e51b service swift] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:47:05.800384 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-0e836f90-fff1-4798-9301-17fd0f41e51b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:47:05.870999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e836f90-fff1-4798-9301-17fd0f41e51b service swift] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:47:05.876777 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fc487-153a-49e3-b1ef-5a01eced71e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:47:05.877329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fc487-153a-49e3-b1ef-5a01eced71e0 service placement] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:47:05.877606 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fc487-153a-49e3-b1ef-5a01eced71e0 service placement] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:47:05.877878 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fc487-153a-49e3-b1ef-5a01eced71e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:47:05.879602 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-1c1fc487-153a-49e3-b1ef-5a01eced71e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:47:05.891216 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1fc487-153a-49e3-b1ef-5a01eced71e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:47:05.892844 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1fc487-153a-49e3-b1ef-5a01eced71e0 service placement] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:47:05.902544 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1098/2230] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:47:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:47:15.898566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=32 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:47:22.743919 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a16d77-65e1-42d5-90a4-859c6381d22c None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:47:22.744564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a16d77-65e1-42d5-90a4-859c6381d22c None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:47:22.744564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a16d77-65e1-42d5-90a4-859c6381d22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:47:22.765865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-c1a16d77-65e1-42d5-90a4-859c6381d22c None None] MFA Rules not processed for user `980157efae8f43b59dc063fda04c0348`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:47:22.767073 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c1a16d77-65e1-42d5-90a4-859c6381d22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:47:22.775131 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1133/2231] 10.222.0.59 () {58 vars in 967 bytes} [Mon Aug 29 22:47:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 22:47:22.798472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fc487-153a-49e3-b1ef-5a01eced71e0 service placement] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:47:22.810917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fc487-153a-49e3-b1ef-5a01eced71e0 service placement] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:47:22.816312 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62349919-bd15-41c5-b267-96d5d20ce7de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f0503ae1b24ce6a8adb3cfd7973f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:47:22.816830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62349919-bd15-41c5-b267-96d5d20ce7de service swift] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:47:22.817107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62349919-bd15-41c5-b267-96d5d20ce7de service swift] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:47:22.817481 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62349919-bd15-41c5-b267-96d5d20ce7de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:47:22.826494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62349919-bd15-41c5-b267-96d5d20ce7de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:47:22.828017 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62349919-bd15-41c5-b267-96d5d20ce7de service swift] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:47:22.837343 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1099/2232] 10.222.0.59 () {60 vars in 1384 bytes} [Mon Aug 29 22:47:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 22:47:29.869551 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df36916-3762-412d-97ed-ccbc03375581 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:47:29.870161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df36916-3762-412d-97ed-ccbc03375581 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:47:29.870161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df36916-3762-412d-97ed-ccbc03375581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:47:29.890174 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-8df36916-3762-412d-97ed-ccbc03375581 None None] MFA Rules not processed for user `9c2fdfdb62b54f05b7d81f14ec197cca`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:47:29.891374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8df36916-3762-412d-97ed-ccbc03375581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:47:29.948700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1134/2233] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:47:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:47:32.834921 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=6 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:47:39.941893 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=28 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:47:48.850619 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de169134-a5a5-4fdc-b03f-b90e749af968 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:47:48.851157 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de169134-a5a5-4fdc-b03f-b90e749af968 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:47:48.851157 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de169134-a5a5-4fdc-b03f-b90e749af968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:47:48.871677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-de169134-a5a5-4fdc-b03f-b90e749af968 None None] MFA Rules not processed for user `484c09f238bd4b0d91b6aeaee913f7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:47:48.872899 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-de169134-a5a5-4fdc-b03f-b90e749af968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:47:48.881285 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1100/2234] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:47:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:47:58.866292 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:15.340256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c598d6-83cf-4d8d-a5ce-2eb0887f801c None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:15.340737 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c598d6-83cf-4d8d-a5ce-2eb0887f801c None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:15.340737 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c598d6-83cf-4d8d-a5ce-2eb0887f801c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:15.363448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-18c598d6-83cf-4d8d-a5ce-2eb0887f801c None None] MFA Rules not processed for user `cc2f5c40aedf47dcb6374d64fcb04add`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:48:15.363856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-18c598d6-83cf-4d8d-a5ce-2eb0887f801c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:15.371315 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1135/2235] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:48:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:48:25.356174 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:45.020666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de169134-a5a5-4fdc-b03f-b90e749af968 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:45.022199 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-de169134-a5a5-4fdc-b03f-b90e749af968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:45.036734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de169134-a5a5-4fdc-b03f-b90e749af968 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:45.043260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325909a-a8d3-4ac0-9652-17a2a78c9c25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:45.043946 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325909a-a8d3-4ac0-9652-17a2a78c9c25 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:45.044296 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325909a-a8d3-4ac0-9652-17a2a78c9c25 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:45.044579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325909a-a8d3-4ac0-9652-17a2a78c9c25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:45.046520 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-7325909a-a8d3-4ac0-9652-17a2a78c9c25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:45.065366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325909a-a8d3-4ac0-9652-17a2a78c9c25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:45.066961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325909a-a8d3-4ac0-9652-17a2a78c9c25 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:45.081611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1101/2236] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:49.296116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c598d6-83cf-4d8d-a5ce-2eb0887f801c None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:49.307363 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c598d6-83cf-4d8d-a5ce-2eb0887f801c None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:49.311690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bebfa5-085e-43b3-8b97-1a415a9ce5b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:49.312287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bebfa5-085e-43b3-8b97-1a415a9ce5b8 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:49.312561 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bebfa5-085e-43b3-8b97-1a415a9ce5b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:49.312827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bebfa5-085e-43b3-8b97-1a415a9ce5b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:49.314613 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-68bebfa5-085e-43b3-8b97-1a415a9ce5b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:49.329231 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bebfa5-085e-43b3-8b97-1a415a9ce5b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:49.330696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bebfa5-085e-43b3-8b97-1a415a9ce5b8 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:49.342526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1136/2237] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:52.736229 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325909a-a8d3-4ac0-9652-17a2a78c9c25 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:52.737778 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-7325909a-a8d3-4ac0-9652-17a2a78c9c25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:52.751801 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325909a-a8d3-4ac0-9652-17a2a78c9c25 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:52.757807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fd3bd-72a4-430c-a505-0e5f00b179e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:52.758314 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fd3bd-72a4-430c-a505-0e5f00b179e1 service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:52.758594 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fd3bd-72a4-430c-a505-0e5f00b179e1 service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:52.758856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fd3bd-72a4-430c-a505-0e5f00b179e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:52.760736 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-8e5fd3bd-72a4-430c-a505-0e5f00b179e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:52.823951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5fd3bd-72a4-430c-a505-0e5f00b179e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:52.825577 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5fd3bd-72a4-430c-a505-0e5f00b179e1 service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:52.839719 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1102/2238] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:48:59.337471 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=6 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:48:59.533271 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bebfa5-085e-43b3-8b97-1a415a9ce5b8 service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:48:59.534640 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-68bebfa5-085e-43b3-8b97-1a415a9ce5b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:59.552976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bebfa5-085e-43b3-8b97-1a415a9ce5b8 service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:48:59.559182 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffab12-a93e-4936-af36-ec8661bebc10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:48:59.559833 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffab12-a93e-4936-af36-ec8661bebc10 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:48:59.560154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffab12-a93e-4936-af36-ec8661bebc10 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:48:59.560548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffab12-a93e-4936-af36-ec8661bebc10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:48:59.562489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-d3ffab12-a93e-4936-af36-ec8661bebc10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:48:59.634148 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffab12-a93e-4936-af36-ec8661bebc10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:48:59.635760 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffab12-a93e-4936-af36-ec8661bebc10 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:48:59.645438 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1137/2239] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:48:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:02.318591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fd3bd-72a4-430c-a505-0e5f00b179e1 service nova] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:02.332331 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fd3bd-72a4-430c-a505-0e5f00b179e1 service nova] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:02.338624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285fc4e1-8cd2-4268-96a7-6fa9158db538 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:02.339256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fc4e1-8cd2-4268-96a7-6fa9158db538 service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:02.339532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fc4e1-8cd2-4268-96a7-6fa9158db538 service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:02.339804 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fc4e1-8cd2-4268-96a7-6fa9158db538 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:02.341683 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-285fc4e1-8cd2-4268-96a7-6fa9158db538 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:02.353574 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285fc4e1-8cd2-4268-96a7-6fa9158db538 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:02.355190 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285fc4e1-8cd2-4268-96a7-6fa9158db538 service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:02.365304 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1103/2240] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:09.640961 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=28 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:12.360724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=40 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:15.302272 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffab12-a93e-4936-af36-ec8661bebc10 service nova] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:15.315895 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffab12-a93e-4936-af36-ec8661bebc10 service nova] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:15.320694 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650e9ea-2c92-4273-9cfc-4a8317210538 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:15.322407 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650e9ea-2c92-4273-9cfc-4a8317210538 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:15.322407 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650e9ea-2c92-4273-9cfc-4a8317210538 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:15.322857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650e9ea-2c92-4273-9cfc-4a8317210538 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:15.325008 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-d650e9ea-2c92-4273-9cfc-4a8317210538 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:15.337319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650e9ea-2c92-4273-9cfc-4a8317210538 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:15.339137 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650e9ea-2c92-4273-9cfc-4a8317210538 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:15.349177 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1138/2241] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:21.542610 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157df36-c9ef-4c69-b7c6-528d8f245e3c None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:21.543457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157df36-c9ef-4c69-b7c6-528d8f245e3c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:21.543654 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157df36-c9ef-4c69-b7c6-528d8f245e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:21.565111 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-0157df36-c9ef-4c69-b7c6-528d8f245e3c None None] MFA Rules not processed for user `43cdaa09925c4b00be3b45cea4fd40b7`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 22:49:21.566521 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-0157df36-c9ef-4c69-b7c6-528d8f245e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:21.574773 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1104/2242] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 22:49:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 22:49:21.908785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650e9ea-2c92-4273-9cfc-4a8317210538 service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:21.922422 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650e9ea-2c92-4273-9cfc-4a8317210538 service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:21.926715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba991e2-7a45-496c-a6f6-64b678cc5577 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:21.927453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba991e2-7a45-496c-a6f6-64b678cc5577 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:21.927792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba991e2-7a45-496c-a6f6-64b678cc5577 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:21.928178 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba991e2-7a45-496c-a6f6-64b678cc5577 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:21.930167 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3ba991e2-7a45-496c-a6f6-64b678cc5577 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:21.940949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba991e2-7a45-496c-a6f6-64b678cc5577 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:21.942385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba991e2-7a45-496c-a6f6-64b678cc5577 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:21.951381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1139/2243] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:22.394322 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157df36-c9ef-4c69-b7c6-528d8f245e3c None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:22.407660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157df36-c9ef-4c69-b7c6-528d8f245e3c None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:22.412836 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db433caf-e8ae-4180-a841-2d01d6a0444b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:22.413618 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db433caf-e8ae-4180-a841-2d01d6a0444b service placement] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:22.414161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db433caf-e8ae-4180-a841-2d01d6a0444b service placement] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:22.414486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db433caf-e8ae-4180-a841-2d01d6a0444b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:22.416604 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-db433caf-e8ae-4180-a841-2d01d6a0444b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:22.429065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db433caf-e8ae-4180-a841-2d01d6a0444b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:22.430959 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db433caf-e8ae-4180-a841-2d01d6a0444b service placement] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:22.441851 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1105/2244] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:23.287909 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba991e2-7a45-496c-a6f6-64b678cc5577 service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:49:23.303605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba991e2-7a45-496c-a6f6-64b678cc5577 service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:49:23.310667 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc98114-b91c-4a2b-aea8-f60cdc307879 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:49:23.311292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc98114-b91c-4a2b-aea8-f60cdc307879 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:49:23.311572 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc98114-b91c-4a2b-aea8-f60cdc307879 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:49:23.311844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc98114-b91c-4a2b-aea8-f60cdc307879 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:49:23.313965 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-9fc98114-b91c-4a2b-aea8-f60cdc307879 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:49:23.329366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc98114-b91c-4a2b-aea8-f60cdc307879 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:49:23.331313 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc98114-b91c-4a2b-aea8-f60cdc307879 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:49:23.345614 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1140/2245] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:49:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:49:32.437075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=9 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:49:33.339376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=18 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:29.519463 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db433caf-e8ae-4180-a841-2d01d6a0444b service placement] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:29.533115 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db433caf-e8ae-4180-a841-2d01d6a0444b service placement] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:29.539969 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8b785-be59-4fa3-81bd-f46a7ed55c93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:29.540623 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8b785-be59-4fa3-81bd-f46a7ed55c93 service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:29.540899 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8b785-be59-4fa3-81bd-f46a7ed55c93 service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:29.541169 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8b785-be59-4fa3-81bd-f46a7ed55c93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:29.543064 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d2a8b785-be59-4fa3-81bd-f46a7ed55c93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:29.555509 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8b785-be59-4fa3-81bd-f46a7ed55c93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:29.557321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8b785-be59-4fa3-81bd-f46a7ed55c93 service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:29.568152 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1106/2246] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:50:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:50:39.563172 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=6 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:50:56.498911 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc98114-b91c-4a2b-aea8-f60cdc307879 service nova] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:50:56.511310 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc98114-b91c-4a2b-aea8-f60cdc307879 service nova] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:50:56.514705 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2af303-2d96-43e4-abaa-2c8e95b8ef82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:50:56.515373 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2af303-2d96-43e4-abaa-2c8e95b8ef82 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:50:56.515669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2af303-2d96-43e4-abaa-2c8e95b8ef82 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:50:56.516003 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2af303-2d96-43e4-abaa-2c8e95b8ef82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:50:56.517960 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6e2af303-2d96-43e4-abaa-2c8e95b8ef82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:50:56.530712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2af303-2d96-43e4-abaa-2c8e95b8ef82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:50:56.532323 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2af303-2d96-43e4-abaa-2c8e95b8ef82 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:50:56.544106 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1141/2247] 10.222.0.59 () {60 vars in 1396 bytes} [Mon Aug 29 22:50:56 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:51:06.538515 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=6 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:51:13.905385 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8b785-be59-4fa3-81bd-f46a7ed55c93 service nova] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:51:13.906853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d2a8b785-be59-4fa3-81bd-f46a7ed55c93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:13.976044 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8b785-be59-4fa3-81bd-f46a7ed55c93 service nova] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:51:13.979946 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074dfa0b-e121-4965-9f7c-3f07a72f4bd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:51:13.980592 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074dfa0b-e121-4965-9f7c-3f07a72f4bd1 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:51:13.980871 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074dfa0b-e121-4965-9f7c-3f07a72f4bd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:51:13.981142 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074dfa0b-e121-4965-9f7c-3f07a72f4bd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:51:13.983147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-074dfa0b-e121-4965-9f7c-3f07a72f4bd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:51:13.998192 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074dfa0b-e121-4965-9f7c-3f07a72f4bd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:51:13.999713 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074dfa0b-e121-4965-9f7c-3f07a72f4bd1 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:51:14.010579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1107/2248] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:51:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:51:24.005719 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=26 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:52:55.874696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2af303-2d96-43e4-abaa-2c8e95b8ef82 service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:52:55.876479 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6e2af303-2d96-43e4-abaa-2c8e95b8ef82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:52:55.981885 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2af303-2d96-43e4-abaa-2c8e95b8ef82 service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:52:55.987572 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f075d-6915-4c97-9804-a4a624446696 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:52:55.988222 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f075d-6915-4c97-9804-a4a624446696 service placement] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:52:55.988498 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f075d-6915-4c97-9804-a4a624446696 service placement] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:52:55.988762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f075d-6915-4c97-9804-a4a624446696 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:52:55.998141 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f075d-6915-4c97-9804-a4a624446696 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:52:55.999741 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f075d-6915-4c97-9804-a4a624446696 service placement] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:52:56.010783 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1142/2249] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:52:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:53:06.006479 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=37 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:53:49.422672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074dfa0b-e121-4965-9f7c-3f07a72f4bd1 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:49.454210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074dfa0b-e121-4965-9f7c-3f07a72f4bd1 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:49.458668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55e8e7-adde-4c2f-8480-815888fc426a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:49.459285 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55e8e7-adde-4c2f-8480-815888fc426a service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:49.459558 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55e8e7-adde-4c2f-8480-815888fc426a service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:49.459826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55e8e7-adde-4c2f-8480-815888fc426a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:49.480223 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed55e8e7-adde-4c2f-8480-815888fc426a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:49.481671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed55e8e7-adde-4c2f-8480-815888fc426a service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:49.504009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1108/2250] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:53:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:53:52.785697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f075d-6915-4c97-9804-a4a624446696 service placement] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:53:52.799054 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f075d-6915-4c97-9804-a4a624446696 service placement] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:53:52.805037 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cba0eb-d9dd-4fd4-b341-4986b4a8e387 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:53:52.805646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cba0eb-d9dd-4fd4-b341-4986b4a8e387 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:53:52.805923 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cba0eb-d9dd-4fd4-b341-4986b4a8e387 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:53:52.806197 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cba0eb-d9dd-4fd4-b341-4986b4a8e387 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:53:52.815706 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cba0eb-d9dd-4fd4-b341-4986b4a8e387 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:53:52.817681 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cba0eb-d9dd-4fd4-b341-4986b4a8e387 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:53:52.827413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1143/2251] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:53:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:53:59.501492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=22 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:02.734791 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55e8e7-adde-4c2f-8480-815888fc426a service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:02.746619 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55e8e7-adde-4c2f-8480-815888fc426a service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:02.753130 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6c4b5-8214-40c6-bb57-e5a6c0d35590 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:02.753751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6c4b5-8214-40c6-bb57-e5a6c0d35590 service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:02.754024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6c4b5-8214-40c6-bb57-e5a6c0d35590 service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:02.754289 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6c4b5-8214-40c6-bb57-e5a6c0d35590 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:02.762996 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f6c4b5-8214-40c6-bb57-e5a6c0d35590 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:02.764627 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f6c4b5-8214-40c6-bb57-e5a6c0d35590 service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:02.796547 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1109/2252] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:54:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:54:02.822921 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=6 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:12.793227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=13 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:54:21.713682 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cba0eb-d9dd-4fd4-b341-4986b4a8e387 service nova] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:54:21.725194 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cba0eb-d9dd-4fd4-b341-4986b4a8e387 service nova] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:54:21.729397 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d4c05-0ef2-4bcf-b42a-bf2079df6188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:54:21.730004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d4c05-0ef2-4bcf-b42a-bf2079df6188 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:54:21.730284 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d4c05-0ef2-4bcf-b42a-bf2079df6188 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:54:21.730548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d4c05-0ef2-4bcf-b42a-bf2079df6188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:54:21.742201 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d4c05-0ef2-4bcf-b42a-bf2079df6188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:54:21.743613 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d4c05-0ef2-4bcf-b42a-bf2079df6188 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:54:21.756496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1144/2253] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:54:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:54:31.750680 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=6 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:55:01.476489 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6c4b5-8214-40c6-bb57-e5a6c0d35590 service nova] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:01.487840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6c4b5-8214-40c6-bb57-e5a6c0d35590 service nova] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:01.491406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af8049-74d7-44fa-ab57-d6c76e108f67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:01.492006 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af8049-74d7-44fa-ab57-d6c76e108f67 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:01.492316 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af8049-74d7-44fa-ab57-d6c76e108f67 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:01.492583 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af8049-74d7-44fa-ab57-d6c76e108f67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:01.500487 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af8049-74d7-44fa-ab57-d6c76e108f67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:01.502013 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af8049-74d7-44fa-ab57-d6c76e108f67 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:01.510676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1110/2254] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:55:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:55:11.506859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=6 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:55:14.791546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d4c05-0ef2-4bcf-b42a-bf2079df6188 service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:14.816764 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d4c05-0ef2-4bcf-b42a-bf2079df6188 service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:14.820572 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d4f89-afdf-4d1a-b4ff-82f888f730be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:14.821178 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d4f89-afdf-4d1a-b4ff-82f888f730be service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:14.821452 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d4f89-afdf-4d1a-b4ff-82f888f730be service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:14.821723 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d4f89-afdf-4d1a-b4ff-82f888f730be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:14.832941 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d4f89-afdf-4d1a-b4ff-82f888f730be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:14.835041 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d4f89-afdf-4d1a-b4ff-82f888f730be service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:14.870595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1145/2255] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:55:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:55:24.866965 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=17 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:55:56.665146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af8049-74d7-44fa-ab57-d6c76e108f67 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:55:56.678583 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af8049-74d7-44fa-ab57-d6c76e108f67 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:55:56.684226 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712639e-1a00-4ffc-919d-bf245c1bfbbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:55:56.684815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712639e-1a00-4ffc-919d-bf245c1bfbbb service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:55:56.685094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712639e-1a00-4ffc-919d-bf245c1bfbbb service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:55:56.685365 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712639e-1a00-4ffc-919d-bf245c1bfbbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:55:56.693515 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712639e-1a00-4ffc-919d-bf245c1bfbbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:55:56.695072 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712639e-1a00-4ffc-919d-bf245c1bfbbb service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:55:56.730935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1111/2256] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:55:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:06.727827 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=13 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:56:15.119174 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d4f89-afdf-4d1a-b4ff-82f888f730be service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:56:15.133265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d4f89-afdf-4d1a-b4ff-82f888f730be service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:56:15.139730 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f2b1e-06f1-487f-b60a-4caac0557c05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:56:15.140373 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f2b1e-06f1-487f-b60a-4caac0557c05 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:56:15.140649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f2b1e-06f1-487f-b60a-4caac0557c05 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:56:15.140929 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f2b1e-06f1-487f-b60a-4caac0557c05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:56:15.209784 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f2b1e-06f1-487f-b60a-4caac0557c05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:56:15.211286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f2b1e-06f1-487f-b60a-4caac0557c05 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:56:15.230630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1146/2257] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:56:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:56:25.227854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=36 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:57:15.372751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712639e-1a00-4ffc-919d-bf245c1bfbbb service nova] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:57:15.382780 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712639e-1a00-4ffc-919d-bf245c1bfbbb service nova] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:57:15.387949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c288b-5e1c-4f2f-b9b6-dca4664b4c12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:57:15.388775 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c288b-5e1c-4f2f-b9b6-dca4664b4c12 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:57:15.389123 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c288b-5e1c-4f2f-b9b6-dca4664b4c12 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:57:15.389485 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c288b-5e1c-4f2f-b9b6-dca4664b4c12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:57:15.402098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c288b-5e1c-4f2f-b9b6-dca4664b4c12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:57:15.403690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c288b-5e1c-4f2f-b9b6-dca4664b4c12 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:57:15.440330 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1112/2258] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:57:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:57:25.436966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=13 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:57:25.912467 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f2b1e-06f1-487f-b60a-4caac0557c05 service nova] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:57:25.998680 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f2b1e-06f1-487f-b60a-4caac0557c05 service nova] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:57:26.005141 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b070c8-67e9-4308-9159-4aafcf3bc314 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:57:26.013586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b070c8-67e9-4308-9159-4aafcf3bc314 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:57:26.013586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b070c8-67e9-4308-9159-4aafcf3bc314 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:57:26.013586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b070c8-67e9-4308-9159-4aafcf3bc314 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:57:26.018395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b070c8-67e9-4308-9159-4aafcf3bc314 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:57:26.020624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b070c8-67e9-4308-9159-4aafcf3bc314 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:57:26.049917 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1147/2259] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:57:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:57:36.031633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=34 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:58:00.853544 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c288b-5e1c-4f2f-b9b6-dca4664b4c12 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:58:00.950844 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c288b-5e1c-4f2f-b9b6-dca4664b4c12 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:58:00.957548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da8072-b30b-4918-b1e2-4be2678a02f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:58:00.958175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da8072-b30b-4918-b1e2-4be2678a02f1 service placement] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:58:00.958447 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da8072-b30b-4918-b1e2-4be2678a02f1 service placement] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:58:00.958715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da8072-b30b-4918-b1e2-4be2678a02f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:58:00.961874 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b6da8072-b30b-4918-b1e2-4be2678a02f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:58:00.981903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da8072-b30b-4918-b1e2-4be2678a02f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:58:00.985424 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da8072-b30b-4918-b1e2-4be2678a02f1 service placement] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:58:01.005872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1113/2260] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:58:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:58:10.997378 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=32 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:59:15.843063 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b070c8-67e9-4308-9159-4aafcf3bc314 service nova] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:59:15.858982 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b070c8-67e9-4308-9159-4aafcf3bc314 service nova] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:59:15.864873 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba5636-4208-4747-a8d7-0d91d7b5a669 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:59:15.865690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba5636-4208-4747-a8d7-0d91d7b5a669 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:59:15.866179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba5636-4208-4747-a8d7-0d91d7b5a669 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:59:15.866491 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba5636-4208-4747-a8d7-0d91d7b5a669 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:59:15.869785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-03ba5636-4208-4747-a8d7-0d91d7b5a669 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:59:15.988817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba5636-4208-4747-a8d7-0d91d7b5a669 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:59:15.990216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba5636-4208-4747-a8d7-0d91d7b5a669 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:59:16.019297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1148/2261] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:59:15 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:59:16.075266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da8072-b30b-4918-b1e2-4be2678a02f1 service placement] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:59:16.087043 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da8072-b30b-4918-b1e2-4be2678a02f1 service placement] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:59:16.091221 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94de20-5cf7-4bcf-9f60-872346237f2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:59:16.091879 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94de20-5cf7-4bcf-9f60-872346237f2c service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:59:16.092180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94de20-5cf7-4bcf-9f60-872346237f2c service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:59:16.092454 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94de20-5cf7-4bcf-9f60-872346237f2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:59:16.094377 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ac94de20-5cf7-4bcf-9f60-872346237f2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:59:16.220660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94de20-5cf7-4bcf-9f60-872346237f2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:59:16.224327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94de20-5cf7-4bcf-9f60-872346237f2c service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:59:16.245414 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1114/2262] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:59:16 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:59:16.406543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba5636-4208-4747-a8d7-0d91d7b5a669 service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:59:16.418221 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba5636-4208-4747-a8d7-0d91d7b5a669 service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:59:16.422175 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb36fea-380a-49fd-9b11-8fce83204cf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:59:16.422793 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb36fea-380a-49fd-9b11-8fce83204cf2 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:59:16.423068 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb36fea-380a-49fd-9b11-8fce83204cf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:59:16.423337 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb36fea-380a-49fd-9b11-8fce83204cf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:59:16.425417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-9eb36fea-380a-49fd-9b11-8fce83204cf2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:59:16.460610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb36fea-380a-49fd-9b11-8fce83204cf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:59:16.463033 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb36fea-380a-49fd-9b11-8fce83204cf2 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:59:16.475703 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1149/2263] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:59:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 22:59:24.427424 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94de20-5cf7-4bcf-9f60-872346237f2c service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 22:59:24.435174 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ac94de20-5cf7-4bcf-9f60-872346237f2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:59:24.479236 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94de20-5cf7-4bcf-9f60-872346237f2c service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 22:59:24.485669 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a97a0-4a1e-4b52-936b-068b8acba6b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 22:59:24.486630 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a97a0-4a1e-4b52-936b-068b8acba6b0 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 22:59:24.487135 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a97a0-4a1e-4b52-936b-068b8acba6b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 22:59:24.487453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a97a0-4a1e-4b52-936b-068b8acba6b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 22:59:24.489935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-023a97a0-4a1e-4b52-936b-068b8acba6b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 22:59:24.519793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a97a0-4a1e-4b52-936b-068b8acba6b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 22:59:24.522535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a97a0-4a1e-4b52-936b-068b8acba6b0 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 22:59:24.542876 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1115/2264] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 22:59:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 22:59:26.469791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=52 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 22:59:34.534189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=44 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:08.814452 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb36fea-380a-49fd-9b11-8fce83204cf2 service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:08.815974 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-9eb36fea-380a-49fd-9b11-8fce83204cf2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:08.906468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb36fea-380a-49fd-9b11-8fce83204cf2 service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:08.912453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e393b6-f5ca-4259-98b6-e4ebdc71a31c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:08.913078 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e393b6-f5ca-4259-98b6-e4ebdc71a31c service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:08.913359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e393b6-f5ca-4259-98b6-e4ebdc71a31c service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:08.913628 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e393b6-f5ca-4259-98b6-e4ebdc71a31c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:08.917046 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a3e393b6-f5ca-4259-98b6-e4ebdc71a31c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:08.936691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e393b6-f5ca-4259-98b6-e4ebdc71a31c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:08.938139 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e393b6-f5ca-4259-98b6-e4ebdc71a31c service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:08.947193 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1150/2265] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:00:15.166667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a97a0-4a1e-4b52-936b-068b8acba6b0 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:15.176072 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a97a0-4a1e-4b52-936b-068b8acba6b0 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:15.180046 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52886cd-5f5e-4469-ab7a-51124e7e3cb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:15.180675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52886cd-5f5e-4469-ab7a-51124e7e3cb3 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:15.180966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52886cd-5f5e-4469-ab7a-51124e7e3cb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:15.181990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52886cd-5f5e-4469-ab7a-51124e7e3cb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:15.184219 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b52886cd-5f5e-4469-ab7a-51124e7e3cb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:00:15.261960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52886cd-5f5e-4469-ab7a-51124e7e3cb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:15.263434 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52886cd-5f5e-4469-ab7a-51124e7e3cb3 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:15.273549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1116/2266] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:00:18.943294 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=31 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:25.268771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=33 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:00:56.283717 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e393b6-f5ca-4259-98b6-e4ebdc71a31c service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:00:56.293749 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e393b6-f5ca-4259-98b6-e4ebdc71a31c service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:00:56.297505 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9cebe-1fb4-4605-9682-5112a3025d54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:00:56.298103 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9cebe-1fb4-4605-9682-5112a3025d54 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:00:56.298380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9cebe-1fb4-4605-9682-5112a3025d54 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:00:56.298649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9cebe-1fb4-4605-9682-5112a3025d54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:00:56.359912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9cebe-1fb4-4605-9682-5112a3025d54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:00:56.361388 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9cebe-1fb4-4605-9682-5112a3025d54 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:00:56.372486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1151/2267] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:00:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:01:06.367642 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=28 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:01:14.677134 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52886cd-5f5e-4469-ab7a-51124e7e3cb3 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:01:14.763884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52886cd-5f5e-4469-ab7a-51124e7e3cb3 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:01:14.768053 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7688b1-f8c7-4e08-80a2-5faf82bcafa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:01:14.769361 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7688b1-f8c7-4e08-80a2-5faf82bcafa2 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:01:14.769649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7688b1-f8c7-4e08-80a2-5faf82bcafa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:01:14.770439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7688b1-f8c7-4e08-80a2-5faf82bcafa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:01:14.772672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ac7688b1-f8c7-4e08-80a2-5faf82bcafa2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:01:14.801349 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7688b1-f8c7-4e08-80a2-5faf82bcafa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:01:14.802880 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7688b1-f8c7-4e08-80a2-5faf82bcafa2 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:01:14.820201 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1117/2268] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:01:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:01:24.809625 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=31 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:03:04.845540 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9cebe-1fb4-4605-9682-5112a3025d54 service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:03:04.847157 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-90e9cebe-1fb4-4605-9682-5112a3025d54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:03:04.928166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9cebe-1fb4-4605-9682-5112a3025d54 service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:03:04.933608 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7708b5-1fd0-4a14-a7ff-485956a061f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:03:04.934216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7708b5-1fd0-4a14-a7ff-485956a061f3 service placement] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:03:04.934492 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7708b5-1fd0-4a14-a7ff-485956a061f3 service placement] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:03:04.934761 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7708b5-1fd0-4a14-a7ff-485956a061f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:03:04.945835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7708b5-1fd0-4a14-a7ff-485956a061f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:03:04.947464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7708b5-1fd0-4a14-a7ff-485956a061f3 service placement] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:03:04.958440 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1152/2269] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:03:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:03:14.953843 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=32 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:03:16.814903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7688b1-f8c7-4e08-80a2-5faf82bcafa2 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:03:16.823402 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7688b1-f8c7-4e08-80a2-5faf82bcafa2 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:03:16.827156 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e759388-3943-4867-a314-57c3db0c29ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:03:16.827746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e759388-3943-4867-a314-57c3db0c29ab service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:03:16.828021 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e759388-3943-4867-a314-57c3db0c29ab service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:03:16.828323 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e759388-3943-4867-a314-57c3db0c29ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:03:16.830137 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6e759388-3943-4867-a314-57c3db0c29ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:03:16.903934 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e759388-3943-4867-a314-57c3db0c29ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:03:16.905404 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e759388-3943-4867-a314-57c3db0c29ab service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:03:16.914691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1118/2270] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:03:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:03:26.910383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=33 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:04:15.808191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7708b5-1fd0-4a14-a7ff-485956a061f3 service placement] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:04:15.810366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-ac7708b5-1fd0-4a14-a7ff-485956a061f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:04:15.855351 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7708b5-1fd0-4a14-a7ff-485956a061f3 service placement] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:04:15.862229 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8d544-cdf5-441b-98ed-f308066e199a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:04:15.863041 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8d544-cdf5-441b-98ed-f308066e199a service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:04:15.863429 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8d544-cdf5-441b-98ed-f308066e199a service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:04:15.863768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8d544-cdf5-441b-98ed-f308066e199a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:04:15.873660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8d544-cdf5-441b-98ed-f308066e199a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:04:15.875247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8d544-cdf5-441b-98ed-f308066e199a service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:04:15.904229 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1153/2271] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:04:15 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:04:16.142010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e759388-3943-4867-a314-57c3db0c29ab service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:04:16.145205 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6e759388-3943-4867-a314-57c3db0c29ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:04:16.180596 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e759388-3943-4867-a314-57c3db0c29ab service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:04:16.187253 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ea518-9d68-49be-abac-06ae0fa2cfb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:04:16.188156 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ea518-9d68-49be-abac-06ae0fa2cfb3 service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:04:16.188438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ea518-9d68-49be-abac-06ae0fa2cfb3 service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:04:16.188708 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ea518-9d68-49be-abac-06ae0fa2cfb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:04:16.198684 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ea518-9d68-49be-abac-06ae0fa2cfb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:04:16.200298 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ea518-9d68-49be-abac-06ae0fa2cfb3 service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:04:16.210791 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1119/2272] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:04:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:04:16.862039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8d544-cdf5-441b-98ed-f308066e199a service nova] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:04:16.863559 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b1d8d544-cdf5-441b-98ed-f308066e199a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:04:16.893932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8d544-cdf5-441b-98ed-f308066e199a service nova] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:04:16.898172 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce666e-16d3-4a47-8752-1c0f84bde55e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:04:16.898787 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce666e-16d3-4a47-8752-1c0f84bde55e service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:04:16.899063 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce666e-16d3-4a47-8752-1c0f84bde55e service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:04:16.899332 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce666e-16d3-4a47-8752-1c0f84bde55e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:04:16.911825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce666e-16d3-4a47-8752-1c0f84bde55e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:04:16.913302 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce666e-16d3-4a47-8752-1c0f84bde55e service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:04:16.928098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1154/2273] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:04:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:04:25.593760 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ea518-9d68-49be-abac-06ae0fa2cfb3 service nova] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:04:25.606057 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ea518-9d68-49be-abac-06ae0fa2cfb3 service nova] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:04:25.613690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d70d6-420f-4a9a-9cd9-3dd3418ce3c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:04:25.614766 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d70d6-420f-4a9a-9cd9-3dd3418ce3c7 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:04:25.615308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d70d6-420f-4a9a-9cd9-3dd3418ce3c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:04:25.615809 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d70d6-420f-4a9a-9cd9-3dd3418ce3c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:04:25.627623 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d70d6-420f-4a9a-9cd9-3dd3418ce3c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:04:25.629072 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d70d6-420f-4a9a-9cd9-3dd3418ce3c7 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:04:25.641968 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1120/2274] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:04:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:04:26.922063 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=35 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:04:35.635399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=17 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:05:13.461495 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce666e-16d3-4a47-8752-1c0f84bde55e service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:05:13.470132 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce666e-16d3-4a47-8752-1c0f84bde55e service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:05:13.474246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dafe8-e0fc-4b4b-a0df-dfffe494a473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:05:13.474863 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dafe8-e0fc-4b4b-a0df-dfffe494a473 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:05:13.475183 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dafe8-e0fc-4b4b-a0df-dfffe494a473 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:05:13.475494 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dafe8-e0fc-4b4b-a0df-dfffe494a473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:05:13.485398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dafe8-e0fc-4b4b-a0df-dfffe494a473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:05:13.487917 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dafe8-e0fc-4b4b-a0df-dfffe494a473 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:05:13.503857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1155/2275] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:05:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:05:17.095076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d70d6-420f-4a9a-9cd9-3dd3418ce3c7 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:05:17.097982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-4d6d70d6-420f-4a9a-9cd9-3dd3418ce3c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:05:17.124911 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d70d6-420f-4a9a-9cd9-3dd3418ce3c7 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:05:17.129852 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b3d58-c7e5-44a3-91d1-11d53cbadc4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:05:17.130758 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b3d58-c7e5-44a3-91d1-11d53cbadc4d service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:05:17.131116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b3d58-c7e5-44a3-91d1-11d53cbadc4d service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:05:17.131457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b3d58-c7e5-44a3-91d1-11d53cbadc4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:05:17.141415 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b3d58-c7e5-44a3-91d1-11d53cbadc4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:05:17.142908 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b3d58-c7e5-44a3-91d1-11d53cbadc4d service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:05:17.172900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1121/2276] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:05:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:05:23.496481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=6 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:05:27.169667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=17 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:07:17.361277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dafe8-e0fc-4b4b-a0df-dfffe494a473 service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:07:17.367676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dafe8-e0fc-4b4b-a0df-dfffe494a473 service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:07:17.371346 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c69ffd-7076-4d35-a7a3-76e62b9ecf44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:07:17.371978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c69ffd-7076-4d35-a7a3-76e62b9ecf44 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:07:17.372505 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c69ffd-7076-4d35-a7a3-76e62b9ecf44 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:07:17.372776 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c69ffd-7076-4d35-a7a3-76e62b9ecf44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:07:17.461639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c69ffd-7076-4d35-a7a3-76e62b9ecf44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:07:17.463122 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c69ffd-7076-4d35-a7a3-76e62b9ecf44 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:07:17.485689 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1156/2277] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:07:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:07:17.542295 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b3d58-c7e5-44a3-91d1-11d53cbadc4d service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:07:17.552533 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b3d58-c7e5-44a3-91d1-11d53cbadc4d service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:07:17.557703 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9a910-534d-4c4c-a8ef-b75a80384c84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:07:17.558538 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9a910-534d-4c4c-a8ef-b75a80384c84 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:07:17.567042 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9a910-534d-4c4c-a8ef-b75a80384c84 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:07:17.567042 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9a910-534d-4c4c-a8ef-b75a80384c84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:07:17.567042 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f9d9a910-534d-4c4c-a8ef-b75a80384c84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:07:17.608357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d9a910-534d-4c4c-a8ef-b75a80384c84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:07:17.610128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d9a910-534d-4c4c-a8ef-b75a80384c84 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:07:17.632990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1122/2278] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:07:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:07:18.084201 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c69ffd-7076-4d35-a7a3-76e62b9ecf44 service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:07:18.097020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c69ffd-7076-4d35-a7a3-76e62b9ecf44 service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:07:18.102912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba25fe-13f4-48df-b9fd-1580f9e953de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:07:18.103645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba25fe-13f4-48df-b9fd-1580f9e953de service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:07:18.103982 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba25fe-13f4-48df-b9fd-1580f9e953de service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:07:18.104643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba25fe-13f4-48df-b9fd-1580f9e953de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:07:18.112058 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-9aba25fe-13f4-48df-b9fd-1580f9e953de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:07:18.125493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba25fe-13f4-48df-b9fd-1580f9e953de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:07:18.127348 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba25fe-13f4-48df-b9fd-1580f9e953de service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:07:18.142991 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1157/2279] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:07:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:07:18.505916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9a910-534d-4c4c-a8ef-b75a80384c84 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:07:18.519573 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9a910-534d-4c4c-a8ef-b75a80384c84 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:07:18.532763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572aae02-023c-4ccb-a369-6367589d2cce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:07:18.533749 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572aae02-023c-4ccb-a369-6367589d2cce service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:07:18.534195 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572aae02-023c-4ccb-a369-6367589d2cce service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:07:18.534879 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572aae02-023c-4ccb-a369-6367589d2cce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:07:18.537923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-572aae02-023c-4ccb-a369-6367589d2cce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:07:18.661595 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572aae02-023c-4ccb-a369-6367589d2cce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:07:18.663871 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572aae02-023c-4ccb-a369-6367589d2cce service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:07:18.699250 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1123/2280] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:07:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:07:21.023073 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bddf0-d462-4567-ab3a-3e852cc72180 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:07:21.023672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bddf0-d462-4567-ab3a-3e852cc72180 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:07:21.023672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bddf0-d462-4567-ab3a-3e852cc72180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:07:21.054480 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-f19bddf0-d462-4567-ab3a-3e852cc72180 None None] MFA Rules not processed for user `42a287a54c4040ac828ee221a3f2e8c7`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:07:21.055846 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f19bddf0-d462-4567-ab3a-3e852cc72180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:07:21.063739 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1158/2281] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:07:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:07:21.467190 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572aae02-023c-4ccb-a369-6367589d2cce service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:07:21.476595 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572aae02-023c-4ccb-a369-6367589d2cce service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:07:21.486810 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8510335-0961-4981-b619-d99192d8b751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:07:21.487447 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8510335-0961-4981-b619-d99192d8b751 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:07:21.489359 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8510335-0961-4981-b619-d99192d8b751 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:07:21.489359 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8510335-0961-4981-b619-d99192d8b751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:07:21.490477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d8510335-0961-4981-b619-d99192d8b751 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:07:21.502215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8510335-0961-4981-b619-d99192d8b751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:07:21.503960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8510335-0961-4981-b619-d99192d8b751 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:07:21.515468 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1124/2282] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:07:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:07:23.044724 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bddf0-d462-4567-ab3a-3e852cc72180 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:07:23.058118 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bddf0-d462-4567-ab3a-3e852cc72180 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:07:23.064011 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e2301-ca95-49f7-98e8-aa21658e0c4e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:07:23.064408 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e2301-ca95-49f7-98e8-aa21658e0c4e service placement] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:07:23.064692 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e2301-ca95-49f7-98e8-aa21658e0c4e service placement] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:07:23.064963 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e2301-ca95-49f7-98e8-aa21658e0c4e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:07:23.071884 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-531e2301-ca95-49f7-98e8-aa21658e0c4e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:07:23.104581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e2301-ca95-49f7-98e8-aa21658e0c4e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:07:23.106861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e2301-ca95-49f7-98e8-aa21658e0c4e service placement] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:07:23.124176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1159/2283] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:07:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:07:24.185096 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ac846-d51d-4226-a95d-eef60354ae32 None None] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:07:24.186374 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ac846-d51d-4226-a95d-eef60354ae32 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:07:24.186374 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ac846-d51d-4226-a95d-eef60354ae32 None None] PATH_INFO: `/` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:07:24.187697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1125/2284] 10.222.0.59 () {54 vars in 921 bytes} [Mon Aug 29 23:07:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 23:07:24.203985 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d5942-3d8f-42c3-bfb4-927f5aafad7a None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:07:24.204552 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d5942-3d8f-42c3-bfb4-927f5aafad7a None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:07:24.204552 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d5942-3d8f-42c3-bfb4-927f5aafad7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:07:24.266947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-3c5d5942-3d8f-42c3-bfb4-927f5aafad7a None None] MFA Rules not processed for user `7215ee41033742d79671f4f42b81cd06`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:07:24.268078 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3c5d5942-3d8f-42c3-bfb4-927f5aafad7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:07:24.277443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1160/2285] 10.222.0.59 () {58 vars in 1018 bytes} [Mon Aug 29 23:07:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:07:24.294539 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ac846-d51d-4226-a95d-eef60354ae32 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:07:24.297380 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-571ac846-d51d-4226-a95d-eef60354ae32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:07:24.332290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ac846-d51d-4226-a95d-eef60354ae32 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:07:24.339895 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409383e-96f5-460f-8508-30d6a43688cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:07:24.340554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409383e-96f5-460f-8508-30d6a43688cb service placement] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:07:24.340824 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409383e-96f5-460f-8508-30d6a43688cb service placement] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:07:24.341090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409383e-96f5-460f-8508-30d6a43688cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:07:24.351704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c409383e-96f5-460f-8508-30d6a43688cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:07:24.353976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c409383e-96f5-460f-8508-30d6a43688cb service placement] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:07:24.384972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1126/2286] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:07:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:07:28.334876 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d5942-3d8f-42c3-bfb4-927f5aafad7a None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:07:28.336544 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3c5d5942-3d8f-42c3-bfb4-927f5aafad7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:07:28.352403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d5942-3d8f-42c3-bfb4-927f5aafad7a None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:07:28.366264 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e3a34-1c68-4a7d-b0a2-33c8785fcbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:07:28.366902 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e3a34-1c68-4a7d-b0a2-33c8785fcbcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:07:28.367181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e3a34-1c68-4a7d-b0a2-33c8785fcbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:07:28.367454 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e3a34-1c68-4a7d-b0a2-33c8785fcbcb admin admin] PATH_INFO: `/v3/users/21feac769b4b48508163f535cdedfbe5` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:07:28.369616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e3a34-1c68-4a7d-b0a2-33c8785fcbcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=21feac769b4b48508163f535cdedfbe5)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:07:28.378807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e3a34-1c68-4a7d-b0a2-33c8785fcbcb admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:07:28.576214 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-e78e3a34-1c68-4a7d-b0a2-33c8785fcbcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21feac769b4b48508163f535cdedfbe5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:07:28.577029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-e78e3a34-1c68-4a7d-b0a2-33c8785fcbcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21feac769b4b48508163f535cdedfbe5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:07:28.584561 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-e78e3a34-1c68-4a7d-b0a2-33c8785fcbcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21feac769b4b48508163f535cdedfbe5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:07:28.587247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:07:28.665449 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-e78e3a34-1c68-4a7d-b0a2-33c8785fcbcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21feac769b4b48508163f535cdedfbe5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:07:28.666076 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-e78e3a34-1c68-4a7d-b0a2-33c8785fcbcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21feac769b4b48508163f535cdedfbe5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:07:28.688339 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1161/2287] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:07:28 2022] DELETE /identity/v3/users/21feac769b4b48508163f535cdedfbe5 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:07:28.966205 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409383e-96f5-460f-8508-30d6a43688cb service placement] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:07:28.978046 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409383e-96f5-460f-8508-30d6a43688cb service placement] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:07:29.056270 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:07:29.063662 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753102c1-f963-4376-84d4-b3d3d949d25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:07:29.064871 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753102c1-f963-4376-84d4-b3d3d949d25a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:07:29.064871 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753102c1-f963-4376-84d4-b3d3d949d25a admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:07:29.070420 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753102c1-f963-4376-84d4-b3d3d949d25a admin admin] PATH_INFO: `/v3/projects/f42dbdfe1c064b2080b48f8d40a4638b` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:07:29.070420 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753102c1-f963-4376-84d4-b3d3d949d25a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f42dbdfe1c064b2080b48f8d40a4638b)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:07:29.070420 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753102c1-f963-4376-84d4-b3d3d949d25a admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:07:29.200722 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-753102c1-f963-4376-84d4-b3d3d949d25a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f42dbdfe1c064b2080b48f8d40a4638b', 'request_id': 'req-753102c1-f963-4376-84d4-b3d3d949d25a'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:07:29.287769 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-753102c1-f963-4376-84d4-b3d3d949d25a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f42dbdfe1c064b2080b48f8d40a4638b', 'request_id': 'req-753102c1-f963-4376-84d4-b3d3d949d25a'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:07:29.294181 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-753102c1-f963-4376-84d4-b3d3d949d25a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f42dbdfe1c064b2080b48f8d40a4638b', 'request_id': 'req-753102c1-f963-4376-84d4-b3d3d949d25a'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:07:29.306913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-753102c1-f963-4376-84d4-b3d3d949d25a admin admin] The token cache is being invalidate because project f42dbdfe1c064b2080b48f8d40a4638b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:07:29.307209 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-753102c1-f963-4376-84d4-b3d3d949d25a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:07:29.307878 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-753102c1-f963-4376-84d4-b3d3d949d25a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:07:29.309546 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1127/2288] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:07:28 2022] DELETE /identity/v3/projects/f42dbdfe1c064b2080b48f8d40a4638b => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:07:29.313774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e3a34-1c68-4a7d-b0a2-33c8785fcbcb admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:07:29.315244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e78e3a34-1c68-4a7d-b0a2-33c8785fcbcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:07:29.330536 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e3a34-1c68-4a7d-b0a2-33c8785fcbcb admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:07:29.396622 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84197d77-2e67-4806-b02e-318f7a03ba71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:07:29.397744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84197d77-2e67-4806-b02e-318f7a03ba71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:07:29.397744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84197d77-2e67-4806-b02e-318f7a03ba71 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:07:29.398044 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84197d77-2e67-4806-b02e-318f7a03ba71 admin admin] PATH_INFO: `/v3/users/e61f970b7a594bec86df1c6bd32259ac` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:07:29.415097 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84197d77-2e67-4806-b02e-318f7a03ba71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e61f970b7a594bec86df1c6bd32259ac)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:07:29.417260 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84197d77-2e67-4806-b02e-318f7a03ba71 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:07:29.659723 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-84197d77-2e67-4806-b02e-318f7a03ba71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e61f970b7a594bec86df1c6bd32259ac'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:07:29.660421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-84197d77-2e67-4806-b02e-318f7a03ba71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e61f970b7a594bec86df1c6bd32259ac'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:07:29.668626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-84197d77-2e67-4806-b02e-318f7a03ba71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e61f970b7a594bec86df1c6bd32259ac'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:07:29.736329 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-84197d77-2e67-4806-b02e-318f7a03ba71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e61f970b7a594bec86df1c6bd32259ac'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:07:29.736956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-84197d77-2e67-4806-b02e-318f7a03ba71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e61f970b7a594bec86df1c6bd32259ac'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:07:29.758831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1162/2289] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:07:29 2022] DELETE /identity/v3/users/e61f970b7a594bec86df1c6bd32259ac => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:07:30.044352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753102c1-f963-4376-84d4-b3d3d949d25a admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:07:30.046686 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-753102c1-f963-4376-84d4-b3d3d949d25a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:07:30.063301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753102c1-f963-4376-84d4-b3d3d949d25a admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:07:30.149538 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f9938-bd8e-42a9-8730-ac6741489722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:07:30.150183 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f9938-bd8e-42a9-8730-ac6741489722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:07:30.150463 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f9938-bd8e-42a9-8730-ac6741489722 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:07:30.150734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f9938-bd8e-42a9-8730-ac6741489722 admin admin] PATH_INFO: `/v3/projects/12c6e5cbb15a47398f6338192abc1949` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:07:30.164038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f9938-bd8e-42a9-8730-ac6741489722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12c6e5cbb15a47398f6338192abc1949)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:07:30.166099 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f9938-bd8e-42a9-8730-ac6741489722 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:07:30.336514 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-757f9938-bd8e-42a9-8730-ac6741489722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12c6e5cbb15a47398f6338192abc1949', 'request_id': 'req-757f9938-bd8e-42a9-8730-ac6741489722'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:07:30.405045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-757f9938-bd8e-42a9-8730-ac6741489722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12c6e5cbb15a47398f6338192abc1949', 'request_id': 'req-757f9938-bd8e-42a9-8730-ac6741489722'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:07:30.411627 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-757f9938-bd8e-42a9-8730-ac6741489722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12c6e5cbb15a47398f6338192abc1949', 'request_id': 'req-757f9938-bd8e-42a9-8730-ac6741489722'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:07:30.424804 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-757f9938-bd8e-42a9-8730-ac6741489722 admin admin] The token cache is being invalidate because project 12c6e5cbb15a47398f6338192abc1949 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:07:30.425073 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-757f9938-bd8e-42a9-8730-ac6741489722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:07:30.425747 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-757f9938-bd8e-42a9-8730-ac6741489722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:07:30.428188 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1128/2290] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:07:30 2022] DELETE /identity/v3/projects/12c6e5cbb15a47398f6338192abc1949 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:07:37.630335 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84197d77-2e67-4806-b02e-318f7a03ba71 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:07:37.631844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-84197d77-2e67-4806-b02e-318f7a03ba71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:07:37.681505 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:07:37.710793 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84197d77-2e67-4806-b02e-318f7a03ba71 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:07:37.784262 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072650b7-62ae-40bb-ac31-3ae98965ca58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:07:37.784926 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072650b7-62ae-40bb-ac31-3ae98965ca58 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:07:37.785222 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072650b7-62ae-40bb-ac31-3ae98965ca58 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:07:37.785516 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072650b7-62ae-40bb-ac31-3ae98965ca58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:07:37.787483 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-072650b7-62ae-40bb-ac31-3ae98965ca58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:07:37.800772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072650b7-62ae-40bb-ac31-3ae98965ca58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:07:37.802372 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072650b7-62ae-40bb-ac31-3ae98965ca58 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:07:37.815013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1163/2291] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:07:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:07:40.420658 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=80,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:07:42.285316 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f9938-bd8e-42a9-8730-ac6741489722 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:07:42.286804 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-757f9938-bd8e-42a9-8730-ac6741489722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:07:42.303353 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f9938-bd8e-42a9-8730-ac6741489722 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:07:42.310551 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae04674-2c94-4058-8204-a62977ffe4e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:07:42.311797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae04674-2c94-4058-8204-a62977ffe4e8 service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:07:42.312093 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae04674-2c94-4058-8204-a62977ffe4e8 service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:07:42.312354 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae04674-2c94-4058-8204-a62977ffe4e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:07:42.314119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-4ae04674-2c94-4058-8204-a62977ffe4e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:07:42.326892 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae04674-2c94-4058-8204-a62977ffe4e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:07:42.328503 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae04674-2c94-4058-8204-a62977ffe4e8 service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:07:42.339853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1129/2292] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:07:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:07:47.809262 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=42 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:07:52.334991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=6 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:08:12.793105 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072650b7-62ae-40bb-ac31-3ae98965ca58 service nova] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:08:12.794488 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-072650b7-62ae-40bb-ac31-3ae98965ca58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:08:12.864947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072650b7-62ae-40bb-ac31-3ae98965ca58 service nova] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:08:12.870652 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396a170-09c7-4be0-8fb7-c32b805810de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:08:12.871286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396a170-09c7-4be0-8fb7-c32b805810de service placement] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:08:12.871558 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396a170-09c7-4be0-8fb7-c32b805810de service placement] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:08:12.871823 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396a170-09c7-4be0-8fb7-c32b805810de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:08:12.873581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-1396a170-09c7-4be0-8fb7-c32b805810de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:08:12.886697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396a170-09c7-4be0-8fb7-c32b805810de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:08:12.888410 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396a170-09c7-4be0-8fb7-c32b805810de service placement] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:08:12.899877 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1164/2293] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:08:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:08:17.126142 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae04674-2c94-4058-8204-a62977ffe4e8 service nova] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:08:17.139042 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae04674-2c94-4058-8204-a62977ffe4e8 service nova] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:08:17.144721 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f692e8-edfe-455e-9a80-c87311378b89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:08:17.145316 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f692e8-edfe-455e-9a80-c87311378b89 service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:08:17.145595 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f692e8-edfe-455e-9a80-c87311378b89 service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:08:17.145869 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f692e8-edfe-455e-9a80-c87311378b89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:08:17.147599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b3f692e8-edfe-455e-9a80-c87311378b89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:08:17.219587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f692e8-edfe-455e-9a80-c87311378b89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:08:17.221253 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f692e8-edfe-455e-9a80-c87311378b89 service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:08:17.240635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1130/2294] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:08:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:08:22.895134 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=32 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:08:27.237083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=36 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:09:16.253863 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396a170-09c7-4be0-8fb7-c32b805810de service placement] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:09:16.255309 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-1396a170-09c7-4be0-8fb7-c32b805810de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:09:16.313992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396a170-09c7-4be0-8fb7-c32b805810de service placement] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:09:16.318789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fde9e2-e763-4f47-b457-ef81a580005c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:09:16.319384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fde9e2-e763-4f47-b457-ef81a580005c service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:09:16.319657 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fde9e2-e763-4f47-b457-ef81a580005c service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:09:16.319924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fde9e2-e763-4f47-b457-ef81a580005c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:09:16.321666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-31fde9e2-e763-4f47-b457-ef81a580005c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:09:16.334722 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fde9e2-e763-4f47-b457-ef81a580005c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:09:16.336153 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fde9e2-e763-4f47-b457-ef81a580005c service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:09:16.346908 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1165/2295] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:09:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:09:17.759304 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f692e8-edfe-455e-9a80-c87311378b89 service nova] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:09:17.759304 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b3f692e8-edfe-455e-9a80-c87311378b89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:09:17.828358 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f692e8-edfe-455e-9a80-c87311378b89 service nova] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:09:17.830493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fde9e2-e763-4f47-b457-ef81a580005c service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:09:17.831914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-31fde9e2-e763-4f47-b457-ef81a580005c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:09:17.833273 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73fe3e-b2f3-4e3b-9912-b6b070f7ac0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:09:17.833898 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73fe3e-b2f3-4e3b-9912-b6b070f7ac0b service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:09:17.834228 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73fe3e-b2f3-4e3b-9912-b6b070f7ac0b service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:09:17.834506 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73fe3e-b2f3-4e3b-9912-b6b070f7ac0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:09:17.836749 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-0d73fe3e-b2f3-4e3b-9912-b6b070f7ac0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:09:17.844503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fde9e2-e763-4f47-b457-ef81a580005c service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:09:17.849358 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144856a3-32e7-4544-be2f-3682c88de892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:09:17.849639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144856a3-32e7-4544-be2f-3682c88de892 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:09:17.849935 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144856a3-32e7-4544-be2f-3682c88de892 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:09:17.850189 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144856a3-32e7-4544-be2f-3682c88de892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:09:17.852064 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-144856a3-32e7-4544-be2f-3682c88de892 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:09:17.943476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73fe3e-b2f3-4e3b-9912-b6b070f7ac0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:09:17.945531 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73fe3e-b2f3-4e3b-9912-b6b070f7ac0b service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:09:17.955605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144856a3-32e7-4544-be2f-3682c88de892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:09:17.957811 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144856a3-32e7-4544-be2f-3682c88de892 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:09:17.958694 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1131/2296] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:09:17 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:09:17.968701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1166/2297] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:09:17 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:09:18.013094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73fe3e-b2f3-4e3b-9912-b6b070f7ac0b service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:09:18.025312 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73fe3e-b2f3-4e3b-9912-b6b070f7ac0b service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:09:18.029452 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97012099-4332-4088-9468-cd3e6e326ba3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:09:18.030165 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97012099-4332-4088-9468-cd3e6e326ba3 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:09:18.030346 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97012099-4332-4088-9468-cd3e6e326ba3 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:09:18.030598 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97012099-4332-4088-9468-cd3e6e326ba3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:09:18.032481 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-97012099-4332-4088-9468-cd3e6e326ba3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:09:18.139130 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97012099-4332-4088-9468-cd3e6e326ba3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:09:18.140820 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97012099-4332-4088-9468-cd3e6e326ba3 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:09:18.160511 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1132/2298] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:09:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:09:23.235673 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a2ef5-4c9b-4115-b379-6bfcd0ec8d7d None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:09:23.236299 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a2ef5-4c9b-4115-b379-6bfcd0ec8d7d None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:09:23.236299 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a2ef5-4c9b-4115-b379-6bfcd0ec8d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:09:23.267844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-276a2ef5-4c9b-4115-b379-6bfcd0ec8d7d None None] MFA Rules not processed for user `9c2fdfdb62b54f05b7d81f14ec197cca`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:09:23.269854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-276a2ef5-4c9b-4115-b379-6bfcd0ec8d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:09:23.277282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1167/2299] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:09:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:09:24.061008 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97012099-4332-4088-9468-cd3e6e326ba3 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:09:24.063351 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-97012099-4332-4088-9468-cd3e6e326ba3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:09:24.078415 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97012099-4332-4088-9468-cd3e6e326ba3 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:09:24.083184 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e63938-30b1-4014-8090-d7d99c8a4f4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:09:24.083805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e63938-30b1-4014-8090-d7d99c8a4f4c service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:09:24.084074 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e63938-30b1-4014-8090-d7d99c8a4f4c service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:09:24.084377 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e63938-30b1-4014-8090-d7d99c8a4f4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:09:24.086512 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f9e63938-30b1-4014-8090-d7d99c8a4f4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:09:24.101118 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:09:24.121127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e63938-30b1-4014-8090-d7d99c8a4f4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:09:24.123028 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e63938-30b1-4014-8090-d7d99c8a4f4c service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:09:24.140229 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1133/2300] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:09:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:09:33.265124 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=65 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:09:34.147860 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=5 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:10:21.450710 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a2ef5-4c9b-4115-b379-6bfcd0ec8d7d None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:10:21.463096 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a2ef5-4c9b-4115-b379-6bfcd0ec8d7d None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:10:21.466777 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8aab76-f7e9-461e-87d4-a02d4c9a858c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:10:21.467366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8aab76-f7e9-461e-87d4-a02d4c9a858c service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:10:21.467637 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8aab76-f7e9-461e-87d4-a02d4c9a858c service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:10:21.467902 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8aab76-f7e9-461e-87d4-a02d4c9a858c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:10:21.469636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6e8aab76-f7e9-461e-87d4-a02d4c9a858c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:10:21.492463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8aab76-f7e9-461e-87d4-a02d4c9a858c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:10:21.493885 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8aab76-f7e9-461e-87d4-a02d4c9a858c service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:10:21.501920 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1168/2301] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:10:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:10:31.498248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=11 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:11:18.433515 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e63938-30b1-4014-8090-d7d99c8a4f4c service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:11:18.445018 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e63938-30b1-4014-8090-d7d99c8a4f4c service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:11:18.449336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c1b73-7bbf-4e11-8eb1-09a872dd2b72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:11:18.454998 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c1b73-7bbf-4e11-8eb1-09a872dd2b72 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:11:18.454998 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c1b73-7bbf-4e11-8eb1-09a872dd2b72 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:11:18.454998 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c1b73-7bbf-4e11-8eb1-09a872dd2b72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:11:18.454998 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-2e7c1b73-7bbf-4e11-8eb1-09a872dd2b72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:11:18.535137 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c1b73-7bbf-4e11-8eb1-09a872dd2b72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:11:18.536734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c1b73-7bbf-4e11-8eb1-09a872dd2b72 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:11:18.547387 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1134/2302] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:11:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:11:23.209431 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b20d7-0bdc-41e6-b773-dbc65dbf0d48 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:11:23.210023 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b20d7-0bdc-41e6-b773-dbc65dbf0d48 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:11:23.210023 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b20d7-0bdc-41e6-b773-dbc65dbf0d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:11:23.229649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-b61b20d7-0bdc-41e6-b773-dbc65dbf0d48 None None] MFA Rules not processed for user `9c2fdfdb62b54f05b7d81f14ec197cca`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:11:23.230839 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b61b20d7-0bdc-41e6-b773-dbc65dbf0d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:11:23.239944 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1169/2303] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:11:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:11:28.542863 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=33 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:11:33.224161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:12:17.604252 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c1b73-7bbf-4e11-8eb1-09a872dd2b72 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:12:17.613765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c1b73-7bbf-4e11-8eb1-09a872dd2b72 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:12:17.618156 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584033e-2efe-4776-9f2d-76acee8c33b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:12:17.618820 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584033e-2efe-4776-9f2d-76acee8c33b7 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:12:17.619098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584033e-2efe-4776-9f2d-76acee8c33b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:12:17.619369 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584033e-2efe-4776-9f2d-76acee8c33b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:12:17.621418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b584033e-2efe-4776-9f2d-76acee8c33b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:12:17.696778 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584033e-2efe-4776-9f2d-76acee8c33b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:12:17.698188 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584033e-2efe-4776-9f2d-76acee8c33b7 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:12:17.707156 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1135/2304] 10.222.0.59 () {60 vars in 1396 bytes} [Mon Aug 29 23:12:17 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:12:27.702964 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=33 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:13:17.308911 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b20d7-0bdc-41e6-b773-dbc65dbf0d48 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:13:17.310440 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b61b20d7-0bdc-41e6-b773-dbc65dbf0d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:13:17.368644 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b20d7-0bdc-41e6-b773-dbc65dbf0d48 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:13:17.374136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd3457-2c03-4b56-9408-edaa0d3805de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:13:17.374795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd3457-2c03-4b56-9408-edaa0d3805de service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:13:17.375068 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd3457-2c03-4b56-9408-edaa0d3805de service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:13:17.375396 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd3457-2c03-4b56-9408-edaa0d3805de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:13:17.383216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd3457-2c03-4b56-9408-edaa0d3805de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:13:17.384881 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd3457-2c03-4b56-9408-edaa0d3805de service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:13:17.394581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1170/2305] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:13:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:13:17.824577 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584033e-2efe-4776-9f2d-76acee8c33b7 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:13:17.826605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b584033e-2efe-4776-9f2d-76acee8c33b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:13:17.890523 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584033e-2efe-4776-9f2d-76acee8c33b7 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:13:17.899179 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6620c9f3-283b-4e31-9994-e4c3c42a73e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:13:17.899806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620c9f3-283b-4e31-9994-e4c3c42a73e8 service placement] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:13:17.900077 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620c9f3-283b-4e31-9994-e4c3c42a73e8 service placement] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:13:17.900380 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620c9f3-283b-4e31-9994-e4c3c42a73e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:13:17.909638 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6620c9f3-283b-4e31-9994-e4c3c42a73e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:13:17.911251 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6620c9f3-283b-4e31-9994-e4c3c42a73e8 service placement] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:13:17.923084 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1136/2306] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:13:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:13:20.486584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd3457-2c03-4b56-9408-edaa0d3805de service nova] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:13:20.500208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd3457-2c03-4b56-9408-edaa0d3805de service nova] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:13:20.505914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69a746-c73d-4f20-b6ee-d294c49ce792 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:13:20.506513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69a746-c73d-4f20-b6ee-d294c49ce792 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:13:20.506787 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69a746-c73d-4f20-b6ee-d294c49ce792 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:13:20.507052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69a746-c73d-4f20-b6ee-d294c49ce792 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:13:20.517669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69a746-c73d-4f20-b6ee-d294c49ce792 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:13:20.519357 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69a746-c73d-4f20-b6ee-d294c49ce792 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:13:20.533340 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1171/2307] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:13:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:13:27.917318 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=25 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:13:30.527894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=29 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:15:19.550487 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6620c9f3-283b-4e31-9994-e4c3c42a73e8 service placement] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:15:19.579913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6620c9f3-283b-4e31-9994-e4c3c42a73e8 service placement] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:15:19.583566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aed244-46b9-4e35-a5fd-6fccb44801d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:15:19.584178 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aed244-46b9-4e35-a5fd-6fccb44801d4 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:19.584459 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aed244-46b9-4e35-a5fd-6fccb44801d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:19.584727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aed244-46b9-4e35-a5fd-6fccb44801d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:19.593051 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aed244-46b9-4e35-a5fd-6fccb44801d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:15:19.594456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aed244-46b9-4e35-a5fd-6fccb44801d4 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:15:19.603178 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1137/2308] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:15:19 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:15:19.651176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69a746-c73d-4f20-b6ee-d294c49ce792 service nova] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:15:19.671551 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69a746-c73d-4f20-b6ee-d294c49ce792 service nova] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:15:19.675146 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb9168-7abc-4e8a-8bdf-e89cbd2ba509 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:15:19.675826 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb9168-7abc-4e8a-8bdf-e89cbd2ba509 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:19.676214 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb9168-7abc-4e8a-8bdf-e89cbd2ba509 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:19.676553 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb9168-7abc-4e8a-8bdf-e89cbd2ba509 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:19.685852 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb9168-7abc-4e8a-8bdf-e89cbd2ba509 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:15:19.687267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb9168-7abc-4e8a-8bdf-e89cbd2ba509 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:15:19.717630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1172/2309] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:15:19 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:15:20.129646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aed244-46b9-4e35-a5fd-6fccb44801d4 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:15:20.149170 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aed244-46b9-4e35-a5fd-6fccb44801d4 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:15:20.152842 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5164f-5e02-4216-b02b-8ff1f8a185e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:15:20.154209 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5164f-5e02-4216-b02b-8ff1f8a185e3 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:20.154492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5164f-5e02-4216-b02b-8ff1f8a185e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:20.155067 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5164f-5e02-4216-b02b-8ff1f8a185e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:20.156594 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-aaf5164f-5e02-4216-b02b-8ff1f8a185e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:15:20.221983 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5164f-5e02-4216-b02b-8ff1f8a185e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:15:20.223417 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5164f-5e02-4216-b02b-8ff1f8a185e3 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:15:20.233453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1138/2310] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:15:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:15:23.784838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb9168-7abc-4e8a-8bdf-e89cbd2ba509 service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:15:23.797116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb9168-7abc-4e8a-8bdf-e89cbd2ba509 service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:15:23.802694 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ec0e5-f794-40a8-aa44-907bf83d7792 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:15:23.803553 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ec0e5-f794-40a8-aa44-907bf83d7792 service placement] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:23.803831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ec0e5-f794-40a8-aa44-907bf83d7792 service placement] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:23.804206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ec0e5-f794-40a8-aa44-907bf83d7792 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:23.806034 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-fb8ec0e5-f794-40a8-aa44-907bf83d7792 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:15:23.818902 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ec0e5-f794-40a8-aa44-907bf83d7792 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:15:23.820663 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ec0e5-f794-40a8-aa44-907bf83d7792 service placement] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:15:23.831521 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1173/2311] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:15:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:15:23.886394 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5164f-5e02-4216-b02b-8ff1f8a185e3 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:15:23.887871 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-aaf5164f-5e02-4216-b02b-8ff1f8a185e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:15:23.901265 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5164f-5e02-4216-b02b-8ff1f8a185e3 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:15:23.906262 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e33e6-066e-4367-9592-9926738b93d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:15:23.906950 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e33e6-066e-4367-9592-9926738b93d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:23.907301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e33e6-066e-4367-9592-9926738b93d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:23.907574 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e33e6-066e-4367-9592-9926738b93d0 admin admin] PATH_INFO: `/v3/users/43cdaa09925c4b00be3b45cea4fd40b7` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:23.908733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e33e6-066e-4367-9592-9926738b93d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43cdaa09925c4b00be3b45cea4fd40b7)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:15:23.910951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e33e6-066e-4367-9592-9926738b93d0 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:15:24.087887 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-850e33e6-066e-4367-9592-9926738b93d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43cdaa09925c4b00be3b45cea4fd40b7'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:15:24.094877 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-850e33e6-066e-4367-9592-9926738b93d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43cdaa09925c4b00be3b45cea4fd40b7'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:15:24.095481 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-850e33e6-066e-4367-9592-9926738b93d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43cdaa09925c4b00be3b45cea4fd40b7'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:15:24.108409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-850e33e6-066e-4367-9592-9926738b93d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43cdaa09925c4b00be3b45cea4fd40b7'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:15:24.109093 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-850e33e6-066e-4367-9592-9926738b93d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43cdaa09925c4b00be3b45cea4fd40b7'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:15:24.176615 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1139/2312] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:15:23 2022] DELETE /identity/v3/users/43cdaa09925c4b00be3b45cea4fd40b7 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:15:24.420256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ec0e5-f794-40a8-aa44-907bf83d7792 service placement] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:15:24.433075 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ec0e5-f794-40a8-aa44-907bf83d7792 service placement] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:15:24.490627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406caed-d178-4056-91f3-c84821bad519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:15:24.491252 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406caed-d178-4056-91f3-c84821bad519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:24.491529 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406caed-d178-4056-91f3-c84821bad519 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:24.491799 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406caed-d178-4056-91f3-c84821bad519 admin admin] PATH_INFO: `/v3/projects/df7ab727f89e4fcb9933c2098b4fb501` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:24.502457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406caed-d178-4056-91f3-c84821bad519 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df7ab727f89e4fcb9933c2098b4fb501)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:15:24.504421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406caed-d178-4056-91f3-c84821bad519 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:15:24.607450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6406caed-d178-4056-91f3-c84821bad519 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df7ab727f89e4fcb9933c2098b4fb501', 'request_id': 'req-6406caed-d178-4056-91f3-c84821bad519'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:15:24.614543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6406caed-d178-4056-91f3-c84821bad519 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df7ab727f89e4fcb9933c2098b4fb501', 'request_id': 'req-6406caed-d178-4056-91f3-c84821bad519'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:15:24.620279 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6406caed-d178-4056-91f3-c84821bad519 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df7ab727f89e4fcb9933c2098b4fb501', 'request_id': 'req-6406caed-d178-4056-91f3-c84821bad519'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:15:24.684587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6406caed-d178-4056-91f3-c84821bad519 admin admin] The token cache is being invalidate because project df7ab727f89e4fcb9933c2098b4fb501 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:15:24.684931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6406caed-d178-4056-91f3-c84821bad519 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:15:24.685562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6406caed-d178-4056-91f3-c84821bad519 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:15:24.687082 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1174/2313] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:15:24 2022] DELETE /identity/v3/projects/df7ab727f89e4fcb9933c2098b4fb501 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:15:24.692843 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe7a05-4e3e-4428-854b-d819ef15072e None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:24.694243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe7a05-4e3e-4428-854b-d819ef15072e None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:24.694243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe7a05-4e3e-4428-854b-d819ef15072e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:24.732609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:15:24.741182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-cffe7a05-4e3e-4428-854b-d819ef15072e None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:15:24.742383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-cffe7a05-4e3e-4428-854b-d819ef15072e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:15:24.803073 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1140/2314] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:15:24.811956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df935e2-3a99-4547-bb82-e9f589fd6d7b None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:24.812585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df935e2-3a99-4547-bb82-e9f589fd6d7b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:24.812585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df935e2-3a99-4547-bb82-e9f589fd6d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:24.832590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-9df935e2-3a99-4547-bb82-e9f589fd6d7b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:15:24.833869 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-9df935e2-3a99-4547-bb82-e9f589fd6d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:15:24.841856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1175/2315] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:15:24.846191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe7a05-4e3e-4428-854b-d819ef15072e None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:15:24.856498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe7a05-4e3e-4428-854b-d819ef15072e None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:15:24.861144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad538b7b-ac2d-4a93-8d2d-d7cae2fc6a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:15:24.861750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad538b7b-ac2d-4a93-8d2d-d7cae2fc6a6a admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:24.862027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad538b7b-ac2d-4a93-8d2d-d7cae2fc6a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:24.862297 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad538b7b-ac2d-4a93-8d2d-d7cae2fc6a6a admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:24.862944 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad538b7b-ac2d-4a93-8d2d-d7cae2fc6a6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:15:24.863305 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad538b7b-ac2d-4a93-8d2d-d7cae2fc6a6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:15:24.864857 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad538b7b-ac2d-4a93-8d2d-d7cae2fc6a6a admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:15:24.880878 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1141/2316] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:15:24.885268 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df935e2-3a99-4547-bb82-e9f589fd6d7b None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:15:24.895753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df935e2-3a99-4547-bb82-e9f589fd6d7b None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:15:24.900470 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5ba6c-ca45-444b-87a5-598acc102737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:15:24.901062 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5ba6c-ca45-444b-87a5-598acc102737 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:24.901342 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5ba6c-ca45-444b-87a5-598acc102737 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:24.901606 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5ba6c-ca45-444b-87a5-598acc102737 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:24.902225 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5ba6c-ca45-444b-87a5-598acc102737 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:15:24.904735 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5ba6c-ca45-444b-87a5-598acc102737 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:15:24.931177 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:15:25.001421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1176/2317] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:15:24 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:15:25.005578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad538b7b-ac2d-4a93-8d2d-d7cae2fc6a6a admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:15:25.016128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad538b7b-ac2d-4a93-8d2d-d7cae2fc6a6a admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:15:25.072341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b714c1-7d6e-4834-ab09-3e5b25fa02e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:15:25.072939 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b714c1-7d6e-4834-ab09-3e5b25fa02e2 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:25.073212 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b714c1-7d6e-4834-ab09-3e5b25fa02e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:25.073478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b714c1-7d6e-4834-ab09-3e5b25fa02e2 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:25.074108 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b714c1-7d6e-4834-ab09-3e5b25fa02e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:15:25.076604 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b714c1-7d6e-4834-ab09-3e5b25fa02e2 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:15:25.194519 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1142/2318] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:15:25 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:15:25.198578 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5ba6c-ca45-444b-87a5-598acc102737 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:15:25.209903 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5ba6c-ca45-444b-87a5-598acc102737 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:15:25.214394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d7eff-5b5d-4be8-bfc6-6c5d8c5f0775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:15:25.215648 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d7eff-5b5d-4be8-bfc6-6c5d8c5f0775 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:25.216242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d7eff-5b5d-4be8-bfc6-6c5d8c5f0775 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:25.216858 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d7eff-5b5d-4be8-bfc6-6c5d8c5f0775 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:25.216982 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d7eff-5b5d-4be8-bfc6-6c5d8c5f0775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:15:25.218574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d7eff-5b5d-4be8-bfc6-6c5d8c5f0775 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:15:25.228876 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1177/2319] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:15:25.233484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b714c1-7d6e-4834-ab09-3e5b25fa02e2 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:15:25.243576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b714c1-7d6e-4834-ab09-3e5b25fa02e2 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:15:25.248365 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824a415-4da3-4d6e-9d45-3e1d990f8c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:15:25.248956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824a415-4da3-4d6e-9d45-3e1d990f8c41 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:25.249228 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824a415-4da3-4d6e-9d45-3e1d990f8c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:25.249599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824a415-4da3-4d6e-9d45-3e1d990f8c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:25.250192 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824a415-4da3-4d6e-9d45-3e1d990f8c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:15:25.251772 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824a415-4da3-4d6e-9d45-3e1d990f8c41 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:15:25.262937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1143/2320] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:15:25.267242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d7eff-5b5d-4be8-bfc6-6c5d8c5f0775 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:15:25.278351 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d7eff-5b5d-4be8-bfc6-6c5d8c5f0775 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:15:25.282899 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14940147-ea16-459d-836e-dedf19ce3fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:15:25.283546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14940147-ea16-459d-836e-dedf19ce3fad admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:25.283897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14940147-ea16-459d-836e-dedf19ce3fad admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:25.284196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14940147-ea16-459d-836e-dedf19ce3fad admin admin] PATH_INFO: `/v3/projects/736ef1b802ce41f88ff3ec717d6af7af/users/8dea86493336414f8e5d84352375b932/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:25.299136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14940147-ea16-459d-836e-dedf19ce3fad admin admin] RBAC: Authorizing `identity:create_grant(project_id=736ef1b802ce41f88ff3ec717d6af7af, user_id=8dea86493336414f8e5d84352375b932, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:15:25.301863 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14940147-ea16-459d-836e-dedf19ce3fad admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:15:25.365208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1178/2321] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:15:25 2022] PUT /identity/v3/projects/736ef1b802ce41f88ff3ec717d6af7af/users/8dea86493336414f8e5d84352375b932/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:15:25.376878 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f03bb-1f77-4100-a6c4-48a035ceb483 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:25.377185 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f03bb-1f77-4100-a6c4-48a035ceb483 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:25.377478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f03bb-1f77-4100-a6c4-48a035ceb483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:25.407730 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-917f03bb-1f77-4100-a6c4-48a035ceb483 None None] MFA Rules not processed for user `8dea86493336414f8e5d84352375b932`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:15:25.409001 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-917f03bb-1f77-4100-a6c4-48a035ceb483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:15:25.470144 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:15:25.475356 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1144/2322] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:15:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:15:25.485210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14940147-ea16-459d-836e-dedf19ce3fad admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:15:25.548191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14940147-ea16-459d-836e-dedf19ce3fad admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:15:25.552809 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8049e7-9273-42c2-aa55-7fb85669f154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:15:25.553404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8049e7-9273-42c2-aa55-7fb85669f154 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:25.553710 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8049e7-9273-42c2-aa55-7fb85669f154 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:25.553943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8049e7-9273-42c2-aa55-7fb85669f154 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:25.554570 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8049e7-9273-42c2-aa55-7fb85669f154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:15:25.556487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8049e7-9273-42c2-aa55-7fb85669f154 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:15:25.635109 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1179/2323] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:15:25 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:15:25.642380 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f03bb-1f77-4100-a6c4-48a035ceb483 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:15:25.653444 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f03bb-1f77-4100-a6c4-48a035ceb483 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:15:25.710300 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac56f69-db82-4174-bfea-75f13970bbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:15:25.710901 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac56f69-db82-4174-bfea-75f13970bbe9 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:25.711176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac56f69-db82-4174-bfea-75f13970bbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:25.711443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac56f69-db82-4174-bfea-75f13970bbe9 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:25.712069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac56f69-db82-4174-bfea-75f13970bbe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:15:25.713983 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac56f69-db82-4174-bfea-75f13970bbe9 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:15:25.834488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1145/2324] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:15:25 2022] POST /identity/v3/users => generated 295 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:15:25.838338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8049e7-9273-42c2-aa55-7fb85669f154 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:15:25.848960 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8049e7-9273-42c2-aa55-7fb85669f154 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:15:25.853535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c15fb-604e-4ad2-a4d0-eeb7c4e6c7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:15:25.854130 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c15fb-604e-4ad2-a4d0-eeb7c4e6c7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:25.854408 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c15fb-604e-4ad2-a4d0-eeb7c4e6c7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:25.854688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c15fb-604e-4ad2-a4d0-eeb7c4e6c7a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:25.855324 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c15fb-604e-4ad2-a4d0-eeb7c4e6c7a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:15:25.857473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c15fb-604e-4ad2-a4d0-eeb7c4e6c7a2 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:15:25.868098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1180/2325] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:15:25.872968 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac56f69-db82-4174-bfea-75f13970bbe9 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:15:25.883016 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac56f69-db82-4174-bfea-75f13970bbe9 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:15:25.887803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe09c4-4cb2-40c0-8eb0-a7907d25a3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:15:25.888477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe09c4-4cb2-40c0-8eb0-a7907d25a3d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:25.888756 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe09c4-4cb2-40c0-8eb0-a7907d25a3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:25.889026 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe09c4-4cb2-40c0-8eb0-a7907d25a3d2 admin admin] PATH_INFO: `/v3/projects/722a6274fdd6462191421bd54b9d26fb/users/15db775436c34078b449a5a7a5d721c5/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:25.903568 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe09c4-4cb2-40c0-8eb0-a7907d25a3d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=722a6274fdd6462191421bd54b9d26fb, user_id=15db775436c34078b449a5a7a5d721c5, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:15:25.906253 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe09c4-4cb2-40c0-8eb0-a7907d25a3d2 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:15:25.994640 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1146/2326] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:15:25 2022] PUT /identity/v3/projects/722a6274fdd6462191421bd54b9d26fb/users/15db775436c34078b449a5a7a5d721c5/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:15:25.999615 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c15fb-604e-4ad2-a4d0-eeb7c4e6c7a2 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:15:26.010629 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c15fb-604e-4ad2-a4d0-eeb7c4e6c7a2 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:15:26.068497 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b20146-b89d-43be-a289-17135cf05c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:15:26.069097 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b20146-b89d-43be-a289-17135cf05c26 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:26.069370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b20146-b89d-43be-a289-17135cf05c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:26.069637 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b20146-b89d-43be-a289-17135cf05c26 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:26.070259 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b20146-b89d-43be-a289-17135cf05c26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:15:26.071840 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b20146-b89d-43be-a289-17135cf05c26 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:15:26.081593 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1181/2327] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:15:26.085888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe09c4-4cb2-40c0-8eb0-a7907d25a3d2 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:15:26.096472 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe09c4-4cb2-40c0-8eb0-a7907d25a3d2 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:15:26.100971 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96649dce-021c-45d1-a172-03af803c15bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:15:26.101611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96649dce-021c-45d1-a172-03af803c15bf admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:26.101885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96649dce-021c-45d1-a172-03af803c15bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:26.102148 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96649dce-021c-45d1-a172-03af803c15bf admin admin] PATH_INFO: `/v3/projects/722a6274fdd6462191421bd54b9d26fb/users/15db775436c34078b449a5a7a5d721c5/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:26.104013 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96649dce-021c-45d1-a172-03af803c15bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=722a6274fdd6462191421bd54b9d26fb, user_id=15db775436c34078b449a5a7a5d721c5, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:15:26.107205 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96649dce-021c-45d1-a172-03af803c15bf admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:15:26.117321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1147/2328] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:15:26 2022] PUT /identity/v3/projects/722a6274fdd6462191421bd54b9d26fb/users/15db775436c34078b449a5a7a5d721c5/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:15:26.128740 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989e766-1d73-4031-9952-38f38e71d0f4 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:26.129020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989e766-1d73-4031-9952-38f38e71d0f4 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:26.129321 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989e766-1d73-4031-9952-38f38e71d0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:26.157061 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-7989e766-1d73-4031-9952-38f38e71d0f4 None None] MFA Rules not processed for user `15db775436c34078b449a5a7a5d721c5`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:15:26.158283 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-7989e766-1d73-4031-9952-38f38e71d0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:15:26.171822 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:15:26.227098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1182/2329] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:15:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:15:27.438130 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96649dce-021c-45d1-a172-03af803c15bf admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:15:27.439920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-96649dce-021c-45d1-a172-03af803c15bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:15:27.498620 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96649dce-021c-45d1-a172-03af803c15bf admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:15:27.502332 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e1b4d-634e-4f64-8c4e-dce96a37b919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:15:27.502938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e1b4d-634e-4f64-8c4e-dce96a37b919 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:15:27.503211 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e1b4d-634e-4f64-8c4e-dce96a37b919 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:15:27.503480 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e1b4d-634e-4f64-8c4e-dce96a37b919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:15:27.505207 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-1b4e1b4d-634e-4f64-8c4e-dce96a37b919 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:15:27.515381 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e1b4d-634e-4f64-8c4e-dce96a37b919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:15:27.516827 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e1b4d-634e-4f64-8c4e-dce96a37b919 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:15:27.524742 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1148/2330] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:15:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:15:36.262563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=22 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:15:37.521232 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=71,keystone:INSERT=5 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:16:26.293190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989e766-1d73-4031-9952-38f38e71d0f4 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:16:26.357272 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989e766-1d73-4031-9952-38f38e71d0f4 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:16:26.361891 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e8e38-0cd9-4e6f-bc43-323ee133edbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:16:26.362824 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e8e38-0cd9-4e6f-bc43-323ee133edbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:26.363285 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e8e38-0cd9-4e6f-bc43-323ee133edbc admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:26.363659 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e8e38-0cd9-4e6f-bc43-323ee133edbc admin admin] PATH_INFO: `/v3/users/8dea86493336414f8e5d84352375b932` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:26.364901 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940e8e38-0cd9-4e6f-bc43-323ee133edbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dea86493336414f8e5d84352375b932)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:16:26.367605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940e8e38-0cd9-4e6f-bc43-323ee133edbc admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:16:26.544370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-940e8e38-0cd9-4e6f-bc43-323ee133edbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dea86493336414f8e5d84352375b932'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:16:26.545082 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-940e8e38-0cd9-4e6f-bc43-323ee133edbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dea86493336414f8e5d84352375b932'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:16:26.551924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-940e8e38-0cd9-4e6f-bc43-323ee133edbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dea86493336414f8e5d84352375b932'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:16:26.643799 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-940e8e38-0cd9-4e6f-bc43-323ee133edbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dea86493336414f8e5d84352375b932'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:16:26.644450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-940e8e38-0cd9-4e6f-bc43-323ee133edbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dea86493336414f8e5d84352375b932'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:16:26.664159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1183/2331] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:16:26 2022] DELETE /identity/v3/users/8dea86493336414f8e5d84352375b932 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:16:26.668748 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af599b2-f214-429c-9c83-38686eb3db4c None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:26.669266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af599b2-f214-429c-9c83-38686eb3db4c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:26.669266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af599b2-f214-429c-9c83-38686eb3db4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:26.688688 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-3af599b2-f214-429c-9c83-38686eb3db4c None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:16:26.689970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3af599b2-f214-429c-9c83-38686eb3db4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:16:26.751615 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1149/2332] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:16:26.761612 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e8e38-0cd9-4e6f-bc43-323ee133edbc admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:16:26.763321 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-940e8e38-0cd9-4e6f-bc43-323ee133edbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:16:26.823584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e8e38-0cd9-4e6f-bc43-323ee133edbc admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:16:26.827273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950a5fe-41a7-406f-9fab-97498adf3cff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:16:26.827881 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950a5fe-41a7-406f-9fab-97498adf3cff service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:26.828183 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950a5fe-41a7-406f-9fab-97498adf3cff service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:26.828463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950a5fe-41a7-406f-9fab-97498adf3cff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:26.838159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950a5fe-41a7-406f-9fab-97498adf3cff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:16:26.839586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950a5fe-41a7-406f-9fab-97498adf3cff service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:16:26.849405 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1184/2333] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:16:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:16:27.285203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af599b2-f214-429c-9c83-38686eb3db4c None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:16:27.286648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3af599b2-f214-429c-9c83-38686eb3db4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:16:27.298832 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af599b2-f214-429c-9c83-38686eb3db4c None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:16:27.303700 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f62ec-03f9-43f7-a23f-aaabffa9a204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:16:27.304861 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f62ec-03f9-43f7-a23f-aaabffa9a204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:27.305143 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f62ec-03f9-43f7-a23f-aaabffa9a204 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:27.305405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f62ec-03f9-43f7-a23f-aaabffa9a204 admin admin] PATH_INFO: `/v3/projects/736ef1b802ce41f88ff3ec717d6af7af` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:27.306600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f62ec-03f9-43f7-a23f-aaabffa9a204 admin admin] RBAC: Authorizing `identity:delete_project(project_id=736ef1b802ce41f88ff3ec717d6af7af)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:16:27.308731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f62ec-03f9-43f7-a23f-aaabffa9a204 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:16:27.470543 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-169f62ec-03f9-43f7-a23f-aaabffa9a204 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '736ef1b802ce41f88ff3ec717d6af7af', 'request_id': 'req-169f62ec-03f9-43f7-a23f-aaabffa9a204'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:16:27.552063 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-169f62ec-03f9-43f7-a23f-aaabffa9a204 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '736ef1b802ce41f88ff3ec717d6af7af', 'request_id': 'req-169f62ec-03f9-43f7-a23f-aaabffa9a204'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:16:27.559543 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-169f62ec-03f9-43f7-a23f-aaabffa9a204 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '736ef1b802ce41f88ff3ec717d6af7af', 'request_id': 'req-169f62ec-03f9-43f7-a23f-aaabffa9a204'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:16:27.572125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-169f62ec-03f9-43f7-a23f-aaabffa9a204 admin admin] The token cache is being invalidate because project 736ef1b802ce41f88ff3ec717d6af7af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:16:27.572506 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-169f62ec-03f9-43f7-a23f-aaabffa9a204 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:16:27.573161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-169f62ec-03f9-43f7-a23f-aaabffa9a204 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:16:27.574649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1150/2334] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:16:27 2022] DELETE /identity/v3/projects/736ef1b802ce41f88ff3ec717d6af7af => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:16:27.578794 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950a5fe-41a7-406f-9fab-97498adf3cff service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:16:27.588945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950a5fe-41a7-406f-9fab-97498adf3cff service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:16:27.607971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:16:27.649370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b17991-7ebc-485c-9025-a1d2cf008f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:16:27.650038 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b17991-7ebc-485c-9025-a1d2cf008f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:27.650317 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b17991-7ebc-485c-9025-a1d2cf008f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:27.650596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b17991-7ebc-485c-9025-a1d2cf008f3e admin admin] PATH_INFO: `/v3/users/15db775436c34078b449a5a7a5d721c5` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:27.651708 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b17991-7ebc-485c-9025-a1d2cf008f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=15db775436c34078b449a5a7a5d721c5)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:16:27.653695 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b17991-7ebc-485c-9025-a1d2cf008f3e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:16:27.873397 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-14b17991-7ebc-485c-9025-a1d2cf008f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15db775436c34078b449a5a7a5d721c5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:16:27.874133 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-14b17991-7ebc-485c-9025-a1d2cf008f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15db775436c34078b449a5a7a5d721c5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:16:27.881236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-14b17991-7ebc-485c-9025-a1d2cf008f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15db775436c34078b449a5a7a5d721c5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:16:27.963961 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-14b17991-7ebc-485c-9025-a1d2cf008f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15db775436c34078b449a5a7a5d721c5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:16:27.964721 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-14b17991-7ebc-485c-9025-a1d2cf008f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15db775436c34078b449a5a7a5d721c5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:16:27.984437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1185/2335] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:16:27 2022] DELETE /identity/v3/users/15db775436c34078b449a5a7a5d721c5 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:16:28.382517 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f62ec-03f9-43f7-a23f-aaabffa9a204 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:16:28.384654 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-169f62ec-03f9-43f7-a23f-aaabffa9a204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:16:28.401770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f62ec-03f9-43f7-a23f-aaabffa9a204 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:16:28.464991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535583e9-d3b6-44de-8fb1-eaa0911664f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:16:28.465655 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535583e9-d3b6-44de-8fb1-eaa0911664f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:28.465940 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535583e9-d3b6-44de-8fb1-eaa0911664f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:28.466191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535583e9-d3b6-44de-8fb1-eaa0911664f3 admin admin] PATH_INFO: `/v3/projects/722a6274fdd6462191421bd54b9d26fb` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:28.467296 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535583e9-d3b6-44de-8fb1-eaa0911664f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=722a6274fdd6462191421bd54b9d26fb)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:16:28.469225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535583e9-d3b6-44de-8fb1-eaa0911664f3 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:16:28.542645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:16:28.559831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-535583e9-d3b6-44de-8fb1-eaa0911664f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '722a6274fdd6462191421bd54b9d26fb', 'request_id': 'req-535583e9-d3b6-44de-8fb1-eaa0911664f3'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:16:28.614692 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-535583e9-d3b6-44de-8fb1-eaa0911664f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '722a6274fdd6462191421bd54b9d26fb', 'request_id': 'req-535583e9-d3b6-44de-8fb1-eaa0911664f3'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:16:28.621091 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-535583e9-d3b6-44de-8fb1-eaa0911664f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '722a6274fdd6462191421bd54b9d26fb', 'request_id': 'req-535583e9-d3b6-44de-8fb1-eaa0911664f3'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:16:28.632639 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-535583e9-d3b6-44de-8fb1-eaa0911664f3 admin admin] The token cache is being invalidate because project 722a6274fdd6462191421bd54b9d26fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:16:28.632921 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-535583e9-d3b6-44de-8fb1-eaa0911664f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:16:28.633645 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-535583e9-d3b6-44de-8fb1-eaa0911664f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:16:28.635124 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1151/2336] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:16:28 2022] DELETE /identity/v3/projects/722a6274fdd6462191421bd54b9d26fb => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:16:28.646890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c341ed5-09f6-4f9b-9bb2-4b311a9511a3 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:28.647411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c341ed5-09f6-4f9b-9bb2-4b311a9511a3 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:28.647781 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c341ed5-09f6-4f9b-9bb2-4b311a9511a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:28.671859 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-7c341ed5-09f6-4f9b-9bb2-4b311a9511a3 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:16:28.673036 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-7c341ed5-09f6-4f9b-9bb2-4b311a9511a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:16:28.732083 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1186/2337] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:16:28.740290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d274c-9adb-4e96-b169-b0d28858fb6f None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:28.740596 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d274c-9adb-4e96-b169-b0d28858fb6f None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:28.740874 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d274c-9adb-4e96-b169-b0d28858fb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:28.761048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-c16d274c-9adb-4e96-b169-b0d28858fb6f None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:16:28.762256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-c16d274c-9adb-4e96-b169-b0d28858fb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:16:28.770177 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1152/2338] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:16:28.775023 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c341ed5-09f6-4f9b-9bb2-4b311a9511a3 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:16:28.784852 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c341ed5-09f6-4f9b-9bb2-4b311a9511a3 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:16:28.790327 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfa42b-0189-43b8-ba65-5235e6e82eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:16:28.790923 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfa42b-0189-43b8-ba65-5235e6e82eef admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:28.791201 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfa42b-0189-43b8-ba65-5235e6e82eef admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:28.791471 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfa42b-0189-43b8-ba65-5235e6e82eef admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:28.792112 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cfa42b-0189-43b8-ba65-5235e6e82eef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:16:28.792431 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cfa42b-0189-43b8-ba65-5235e6e82eef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:16:28.794017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cfa42b-0189-43b8-ba65-5235e6e82eef admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:16:28.805647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1187/2339] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:16:28.809933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d274c-9adb-4e96-b169-b0d28858fb6f None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:16:28.819253 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d274c-9adb-4e96-b169-b0d28858fb6f None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:16:28.824099 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03737683-470e-402b-899d-753fa09a47b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:16:28.824733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03737683-470e-402b-899d-753fa09a47b7 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:28.825003 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03737683-470e-402b-899d-753fa09a47b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:28.825269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03737683-470e-402b-899d-753fa09a47b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:28.825890 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03737683-470e-402b-899d-753fa09a47b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:16:28.827769 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03737683-470e-402b-899d-753fa09a47b7 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:16:28.937037 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1153/2340] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:16:28 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:16:28.941208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfa42b-0189-43b8-ba65-5235e6e82eef admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:16:28.951390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfa42b-0189-43b8-ba65-5235e6e82eef admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:16:28.979040 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:16:29.010132 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59c433-10d7-4203-b31a-c0076e590b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:16:29.010730 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59c433-10d7-4203-b31a-c0076e590b5c admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:29.011006 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59c433-10d7-4203-b31a-c0076e590b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:29.011273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59c433-10d7-4203-b31a-c0076e590b5c admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:29.011898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59c433-10d7-4203-b31a-c0076e590b5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:16:29.013930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59c433-10d7-4203-b31a-c0076e590b5c admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:16:29.118396 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1188/2341] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:16:28 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:16:29.123042 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03737683-470e-402b-899d-753fa09a47b7 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:16:29.134405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03737683-470e-402b-899d-753fa09a47b7 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:16:29.139147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b7bc5-44a3-4a9c-8ddc-f92ac1af43b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:16:29.139746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b7bc5-44a3-4a9c-8ddc-f92ac1af43b3 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:29.140021 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b7bc5-44a3-4a9c-8ddc-f92ac1af43b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:29.140319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b7bc5-44a3-4a9c-8ddc-f92ac1af43b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:29.140956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b7bc5-44a3-4a9c-8ddc-f92ac1af43b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:16:29.142542 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b7bc5-44a3-4a9c-8ddc-f92ac1af43b3 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:16:29.152843 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1154/2342] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:16:29.157006 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59c433-10d7-4203-b31a-c0076e590b5c admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:16:29.168084 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59c433-10d7-4203-b31a-c0076e590b5c admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:16:29.172797 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c90e7-4d71-492f-b538-be5eb8e8b4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:16:29.173395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c90e7-4d71-492f-b538-be5eb8e8b4bc admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:29.173673 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c90e7-4d71-492f-b538-be5eb8e8b4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:29.173943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c90e7-4d71-492f-b538-be5eb8e8b4bc admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:29.174582 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c90e7-4d71-492f-b538-be5eb8e8b4bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:16:29.176200 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c90e7-4d71-492f-b538-be5eb8e8b4bc admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:16:29.186866 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1189/2343] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:16:29.191060 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b7bc5-44a3-4a9c-8ddc-f92ac1af43b3 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:16:29.201010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b7bc5-44a3-4a9c-8ddc-f92ac1af43b3 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:16:29.205515 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e2c33-ca27-4977-bc35-aaf507bb620f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:16:29.206154 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e2c33-ca27-4977-bc35-aaf507bb620f admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:29.206427 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e2c33-ca27-4977-bc35-aaf507bb620f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:29.206694 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e2c33-ca27-4977-bc35-aaf507bb620f admin admin] PATH_INFO: `/v3/projects/4e4d6cd42d224f1d88ba9c7736727482/users/2288d88eb89c4c798007acd9974c6bf0/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:29.220969 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e2c33-ca27-4977-bc35-aaf507bb620f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e4d6cd42d224f1d88ba9c7736727482, user_id=2288d88eb89c4c798007acd9974c6bf0, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:16:29.223580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e2c33-ca27-4977-bc35-aaf507bb620f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:16:29.299768 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1155/2344] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:16:29 2022] PUT /identity/v3/projects/4e4d6cd42d224f1d88ba9c7736727482/users/2288d88eb89c4c798007acd9974c6bf0/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:16:29.311285 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e9d4c-cb21-4ce0-9507-f5a4d34fb516 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:29.311569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e9d4c-cb21-4ce0-9507-f5a4d34fb516 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:29.311839 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e9d4c-cb21-4ce0-9507-f5a4d34fb516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:29.340555 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-5a7e9d4c-cb21-4ce0-9507-f5a4d34fb516 None None] MFA Rules not processed for user `2288d88eb89c4c798007acd9974c6bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:16:29.341753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-5a7e9d4c-cb21-4ce0-9507-f5a4d34fb516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:16:29.405328 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1190/2345] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:16:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:16:29.409821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e2c33-ca27-4977-bc35-aaf507bb620f admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:16:29.472535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e2c33-ca27-4977-bc35-aaf507bb620f admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:16:29.478139 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22cf53e-6502-4725-8f0f-74dda74e853d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:16:29.478739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cf53e-6502-4725-8f0f-74dda74e853d admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:29.479012 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cf53e-6502-4725-8f0f-74dda74e853d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:29.479397 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cf53e-6502-4725-8f0f-74dda74e853d admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:29.480083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22cf53e-6502-4725-8f0f-74dda74e853d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:16:29.481970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22cf53e-6502-4725-8f0f-74dda74e853d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:16:29.577015 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1156/2346] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:16:29 2022] POST /identity/v3/projects => generated 387 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:16:29.580997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e9d4c-cb21-4ce0-9507-f5a4d34fb516 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:16:29.592738 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e9d4c-cb21-4ce0-9507-f5a4d34fb516 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:16:29.650342 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ebf0e-c3f9-41fd-b3c5-7c7722e22910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:16:29.651118 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ebf0e-c3f9-41fd-b3c5-7c7722e22910 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:29.651236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ebf0e-c3f9-41fd-b3c5-7c7722e22910 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:29.651458 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ebf0e-c3f9-41fd-b3c5-7c7722e22910 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:29.652081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ebf0e-c3f9-41fd-b3c5-7c7722e22910 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:16:29.654025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ebf0e-c3f9-41fd-b3c5-7c7722e22910 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:16:29.758446 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1191/2347] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:16:29 2022] POST /identity/v3/users => generated 300 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:16:29.762543 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22cf53e-6502-4725-8f0f-74dda74e853d admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:16:29.773100 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22cf53e-6502-4725-8f0f-74dda74e853d admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:16:29.778252 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb11fd-7fec-4443-8806-e078eea683fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:16:29.778848 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb11fd-7fec-4443-8806-e078eea683fd admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:29.779185 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb11fd-7fec-4443-8806-e078eea683fd admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:29.779456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb11fd-7fec-4443-8806-e078eea683fd admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:29.780083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb11fd-7fec-4443-8806-e078eea683fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:16:29.781688 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb11fd-7fec-4443-8806-e078eea683fd admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:16:29.793320 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1157/2348] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:16:29.803310 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ebf0e-c3f9-41fd-b3c5-7c7722e22910 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:16:29.813586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ebf0e-c3f9-41fd-b3c5-7c7722e22910 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:16:29.818234 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd6c69-db34-477f-8cbb-70afb5f293dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:16:29.818876 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd6c69-db34-477f-8cbb-70afb5f293dd admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:29.819190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd6c69-db34-477f-8cbb-70afb5f293dd admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:29.819425 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd6c69-db34-477f-8cbb-70afb5f293dd admin admin] PATH_INFO: `/v3/projects/96732f1eeaf64fc8a23e8478c0c2f4c6/users/2a86212007c044f6b3feade9646d0038/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:29.835284 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd6c69-db34-477f-8cbb-70afb5f293dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=96732f1eeaf64fc8a23e8478c0c2f4c6, user_id=2a86212007c044f6b3feade9646d0038, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:16:29.838508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd6c69-db34-477f-8cbb-70afb5f293dd admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:16:29.912282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1192/2349] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:16:29 2022] PUT /identity/v3/projects/96732f1eeaf64fc8a23e8478c0c2f4c6/users/2a86212007c044f6b3feade9646d0038/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:16:29.916689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb11fd-7fec-4443-8806-e078eea683fd admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:16:29.927069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb11fd-7fec-4443-8806-e078eea683fd admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:16:29.948463 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:16:29.984838 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0010a1b-f87f-4694-9366-8e0b86901bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:16:29.985457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0010a1b-f87f-4694-9366-8e0b86901bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:29.985735 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0010a1b-f87f-4694-9366-8e0b86901bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:29.986010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0010a1b-f87f-4694-9366-8e0b86901bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:29.986647 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0010a1b-f87f-4694-9366-8e0b86901bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:16:29.988278 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0010a1b-f87f-4694-9366-8e0b86901bb4 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:16:29.998015 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1158/2350] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:16:30.002277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd6c69-db34-477f-8cbb-70afb5f293dd admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:16:30.008200 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:16:30.014386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd6c69-db34-477f-8cbb-70afb5f293dd admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:16:30.018928 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c2621-994f-4bbc-8d6d-2d508f86080c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:16:30.019577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c2621-994f-4bbc-8d6d-2d508f86080c admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:30.019852 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c2621-994f-4bbc-8d6d-2d508f86080c admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:30.020116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c2621-994f-4bbc-8d6d-2d508f86080c admin admin] PATH_INFO: `/v3/projects/96732f1eeaf64fc8a23e8478c0c2f4c6/users/2a86212007c044f6b3feade9646d0038/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:30.022017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c2621-994f-4bbc-8d6d-2d508f86080c admin admin] RBAC: Authorizing `identity:create_grant(project_id=96732f1eeaf64fc8a23e8478c0c2f4c6, user_id=2a86212007c044f6b3feade9646d0038, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:16:30.025424 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c2621-994f-4bbc-8d6d-2d508f86080c admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:16:30.035586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1193/2351] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:16:30 2022] PUT /identity/v3/projects/96732f1eeaf64fc8a23e8478c0c2f4c6/users/2a86212007c044f6b3feade9646d0038/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:16:30.047719 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881958d-276b-45a8-aede-1f4a7e56b5e0 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:16:30.048026 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881958d-276b-45a8-aede-1f4a7e56b5e0 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:16:30.048324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881958d-276b-45a8-aede-1f4a7e56b5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:16:30.077692 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-e881958d-276b-45a8-aede-1f4a7e56b5e0 None None] MFA Rules not processed for user `2a86212007c044f6b3feade9646d0038`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:16:30.078955 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-e881958d-276b-45a8-aede-1f4a7e56b5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:16:30.148634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1159/2352] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:16:30 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:16:40.140222 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=4,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:16:40.163566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=52 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:17:20.124496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c2621-994f-4bbc-8d6d-2d508f86080c admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:17:20.126407 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-984c2621-994f-4bbc-8d6d-2d508f86080c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:17:20.196273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c2621-994f-4bbc-8d6d-2d508f86080c admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:17:20.199865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166f0c2-90c4-4382-a997-62927e957523 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:17:20.200506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166f0c2-90c4-4382-a997-62927e957523 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:20.200783 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166f0c2-90c4-4382-a997-62927e957523 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:20.201105 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166f0c2-90c4-4382-a997-62927e957523 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:20.202772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e166f0c2-90c4-4382-a997-62927e957523 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:17:20.269944 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166f0c2-90c4-4382-a997-62927e957523 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:17:20.271374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166f0c2-90c4-4382-a997-62927e957523 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:17:20.289757 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1194/2353] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:17:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:17:20.412161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881958d-276b-45a8-aede-1f4a7e56b5e0 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:17:20.422074 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881958d-276b-45a8-aede-1f4a7e56b5e0 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:17:20.425636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3cbe6-43e5-4ed4-8da2-f21fc56643a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:17:20.426240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3cbe6-43e5-4ed4-8da2-f21fc56643a4 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:20.426511 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3cbe6-43e5-4ed4-8da2-f21fc56643a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:20.426776 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3cbe6-43e5-4ed4-8da2-f21fc56643a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:20.428502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-28a3cbe6-43e5-4ed4-8da2-f21fc56643a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:17:20.441047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application [None req-28a3cbe6-43e5-4ed4-8da2-f21fc56643a4 service cinder] Could not find project: 736ef1b802ce41f88ff3ec717d6af7af.: keystone.exception.ProjectNotFound: Could not find project: 736ef1b802ce41f88ff3ec717d6af7af. Aug 29 23:17:20.441047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:17:20.441047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:17:20.441047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:17:20.441047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:17:20.441047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:17:20.441047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:17:20.441047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:17:20.441047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:17:20.441047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:17:20.441047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:17:20.441047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:17:20.441047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Aug 29 23:17:20.441047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Aug 29 23:17:20.441047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Aug 29 23:17:20.442581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Aug 29 23:17:20.442581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Aug 29 23:17:20.442581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Aug 29 23:17:20.442581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:17:20.442581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:17:20.442581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Aug 29 23:17:20.442581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Aug 29 23:17:20.442581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:17:20.442581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:17:20.442581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:17:20.442581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:17:20.442581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:17:20.442581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 23:17:20.442581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:17:20.442581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:17:20.442581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:17:20.442581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:17:20.444090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:17:20.444090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:17:20.444090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:17:20.444090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:17:20.444090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Aug 29 23:17:20.444090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Aug 29 23:17:20.444090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Aug 29 23:17:20.444090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self._validate_token_resources() Aug 29 23:17:20.444090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Aug 29 23:17:20.444090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Aug 29 23:17:20.444090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Aug 29 23:17:20.444090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Aug 29 23:17:20.444090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:17:20.444090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:17:20.444090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:17:20.444090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:17:20.444090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:17:20.445737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:17:20.445737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:17:20.445737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application with Lock( Aug 29 23:17:20.445737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:17:20.445737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:17:20.445737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:17:20.445737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:17:20.445737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:17:20.445737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:17:20.445737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:17:20.445737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:17:20.445737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:17:20.445737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:17:20.445737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:17:20.445737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:17:20.445737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:17:20.445737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:17:20.447158 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 736ef1b802ce41f88ff3ec717d6af7af. Aug 29 23:17:20.447158 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: ERROR keystone.server.flask.application Aug 29 23:17:20.447158 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1160/2354] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:17:20 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 31 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 29 23:17:20.626397 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166f0c2-90c4-4382-a997-62927e957523 service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:17:20.628173 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e166f0c2-90c4-4382-a997-62927e957523 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:17:20.649165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166f0c2-90c4-4382-a997-62927e957523 service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:17:20.666918 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14bc77-ca63-413a-ae45-8062e4ff2ee9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:17:20.667540 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14bc77-ca63-413a-ae45-8062e4ff2ee9 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:20.667811 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14bc77-ca63-413a-ae45-8062e4ff2ee9 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:20.668082 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14bc77-ca63-413a-ae45-8062e4ff2ee9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:20.680881 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d14bc77-ca63-413a-ae45-8062e4ff2ee9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:17:20.682384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d14bc77-ca63-413a-ae45-8062e4ff2ee9 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:17:20.701960 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1195/2355] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:17:20 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:17:20.749343 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3cbe6-43e5-4ed4-8da2-f21fc56643a4 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:17:20.750750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-28a3cbe6-43e5-4ed4-8da2-f21fc56643a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:17:20.809713 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3cbe6-43e5-4ed4-8da2-f21fc56643a4 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:17:20.813485 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4615ba-2a07-4dc3-89de-e133dbbba7f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:17:20.814119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4615ba-2a07-4dc3-89de-e133dbbba7f1 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:20.814402 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4615ba-2a07-4dc3-89de-e133dbbba7f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:20.814671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4615ba-2a07-4dc3-89de-e133dbbba7f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:20.816452 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-1a4615ba-2a07-4dc3-89de-e133dbbba7f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:17:20.899334 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4615ba-2a07-4dc3-89de-e133dbbba7f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:17:20.900789 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4615ba-2a07-4dc3-89de-e133dbbba7f1 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:17:20.922594 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1161/2356] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:17:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:17:24.569289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14bc77-ca63-413a-ae45-8062e4ff2ee9 service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:17:24.570738 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-2d14bc77-ca63-413a-ae45-8062e4ff2ee9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:17:24.587080 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14bc77-ca63-413a-ae45-8062e4ff2ee9 service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:17:24.591745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ed12b-b888-4b17-9dfa-b40fe4f6ec06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:17:24.592426 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ed12b-b888-4b17-9dfa-b40fe4f6ec06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:24.592701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ed12b-b888-4b17-9dfa-b40fe4f6ec06 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:24.592966 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ed12b-b888-4b17-9dfa-b40fe4f6ec06 admin admin] PATH_INFO: `/v3/users/42a287a54c4040ac828ee221a3f2e8c7` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:24.594071 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ed12b-b888-4b17-9dfa-b40fe4f6ec06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42a287a54c4040ac828ee221a3f2e8c7)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:17:24.595992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ed12b-b888-4b17-9dfa-b40fe4f6ec06 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:17:24.764629 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-628ed12b-b888-4b17-9dfa-b40fe4f6ec06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42a287a54c4040ac828ee221a3f2e8c7'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:17:24.765290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-628ed12b-b888-4b17-9dfa-b40fe4f6ec06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42a287a54c4040ac828ee221a3f2e8c7'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:17:24.772976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-628ed12b-b888-4b17-9dfa-b40fe4f6ec06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42a287a54c4040ac828ee221a3f2e8c7'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:17:24.863782 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-628ed12b-b888-4b17-9dfa-b40fe4f6ec06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42a287a54c4040ac828ee221a3f2e8c7'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:17:24.864451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-628ed12b-b888-4b17-9dfa-b40fe4f6ec06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42a287a54c4040ac828ee221a3f2e8c7'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:17:24.869556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:17:24.883387 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1196/2357] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:17:24 2022] DELETE /identity/v3/users/42a287a54c4040ac828ee221a3f2e8c7 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:17:25.157621 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4615ba-2a07-4dc3-89de-e133dbbba7f1 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:17:25.168223 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4615ba-2a07-4dc3-89de-e133dbbba7f1 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:17:25.229525 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706bc2e-038a-4f81-8539-027743d7b772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:17:25.230160 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706bc2e-038a-4f81-8539-027743d7b772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:25.230434 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706bc2e-038a-4f81-8539-027743d7b772 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:25.230706 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706bc2e-038a-4f81-8539-027743d7b772 admin admin] PATH_INFO: `/v3/projects/d45f8dd557e14a6483ebe7e39d902641` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:25.232541 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706bc2e-038a-4f81-8539-027743d7b772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d45f8dd557e14a6483ebe7e39d902641)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:17:25.234465 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706bc2e-038a-4f81-8539-027743d7b772 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:17:25.250788 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:17:25.385359 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5706bc2e-038a-4f81-8539-027743d7b772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd45f8dd557e14a6483ebe7e39d902641', 'request_id': 'req-5706bc2e-038a-4f81-8539-027743d7b772'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:17:25.459145 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5706bc2e-038a-4f81-8539-027743d7b772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd45f8dd557e14a6483ebe7e39d902641', 'request_id': 'req-5706bc2e-038a-4f81-8539-027743d7b772'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:17:25.466411 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5706bc2e-038a-4f81-8539-027743d7b772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd45f8dd557e14a6483ebe7e39d902641', 'request_id': 'req-5706bc2e-038a-4f81-8539-027743d7b772'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:17:25.477629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5706bc2e-038a-4f81-8539-027743d7b772 admin admin] The token cache is being invalidate because project d45f8dd557e14a6483ebe7e39d902641 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:17:25.477919 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5706bc2e-038a-4f81-8539-027743d7b772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:17:25.478559 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5706bc2e-038a-4f81-8539-027743d7b772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:17:25.480039 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1162/2358] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:17:25 2022] DELETE /identity/v3/projects/d45f8dd557e14a6483ebe7e39d902641 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:17:25.486401 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f74e9b-8f06-4b2d-8cb8-03b06550b855 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:25.486683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f74e9b-8f06-4b2d-8cb8-03b06550b855 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:25.486952 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f74e9b-8f06-4b2d-8cb8-03b06550b855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:25.516432 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-a6f74e9b-8f06-4b2d-8cb8-03b06550b855 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:17:25.517620 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a6f74e9b-8f06-4b2d-8cb8-03b06550b855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:17:25.580740 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1197/2359] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:17:25.588679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984adc5-3a8b-498e-84d3-b46beabb155a None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:25.588972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984adc5-3a8b-498e-84d3-b46beabb155a None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:25.589247 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984adc5-3a8b-498e-84d3-b46beabb155a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:25.610160 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-2984adc5-3a8b-498e-84d3-b46beabb155a None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:17:25.611374 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-2984adc5-3a8b-498e-84d3-b46beabb155a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:17:25.619438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1163/2360] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:17:25.623825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f74e9b-8f06-4b2d-8cb8-03b06550b855 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:17:25.634138 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f74e9b-8f06-4b2d-8cb8-03b06550b855 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:17:25.639772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448a9c3-c9f7-4868-a9f0-f5493b2a2f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:17:25.640344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448a9c3-c9f7-4868-a9f0-f5493b2a2f38 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:25.640624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448a9c3-c9f7-4868-a9f0-f5493b2a2f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:25.640895 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448a9c3-c9f7-4868-a9f0-f5493b2a2f38 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:25.641540 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448a9c3-c9f7-4868-a9f0-f5493b2a2f38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:17:25.641839 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448a9c3-c9f7-4868-a9f0-f5493b2a2f38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:17:25.643696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448a9c3-c9f7-4868-a9f0-f5493b2a2f38 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:17:25.656045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1198/2361] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:17:25.660298 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984adc5-3a8b-498e-84d3-b46beabb155a None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:17:25.670042 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984adc5-3a8b-498e-84d3-b46beabb155a None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:17:25.674746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c28062-9baa-41c0-bea6-b4d52f17079b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:17:25.675385 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c28062-9baa-41c0-bea6-b4d52f17079b admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:25.675661 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c28062-9baa-41c0-bea6-b4d52f17079b admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:25.675927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c28062-9baa-41c0-bea6-b4d52f17079b admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:25.676612 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c28062-9baa-41c0-bea6-b4d52f17079b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:17:25.678491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c28062-9baa-41c0-bea6-b4d52f17079b admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:17:25.764096 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1164/2362] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:17:25 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:17:25.768327 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448a9c3-c9f7-4868-a9f0-f5493b2a2f38 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:17:25.779238 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448a9c3-c9f7-4868-a9f0-f5493b2a2f38 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:17:25.834376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2389e7d-afe9-4f86-a9d7-4b2eec2fcfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:17:25.834967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2389e7d-afe9-4f86-a9d7-4b2eec2fcfe1 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:25.835240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2389e7d-afe9-4f86-a9d7-4b2eec2fcfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:25.835502 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2389e7d-afe9-4f86-a9d7-4b2eec2fcfe1 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:25.836111 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2389e7d-afe9-4f86-a9d7-4b2eec2fcfe1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:17:25.838036 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2389e7d-afe9-4f86-a9d7-4b2eec2fcfe1 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:17:25.951715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1199/2363] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:17:25 2022] POST /identity/v3/users => generated 285 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:17:25.956091 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c28062-9baa-41c0-bea6-b4d52f17079b admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:17:25.968234 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c28062-9baa-41c0-bea6-b4d52f17079b admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:17:25.973173 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68545296-a362-4da9-bbd0-e91dc17c1676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:17:25.973839 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68545296-a362-4da9-bbd0-e91dc17c1676 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:25.974160 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68545296-a362-4da9-bbd0-e91dc17c1676 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:25.974436 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68545296-a362-4da9-bbd0-e91dc17c1676 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:25.975074 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68545296-a362-4da9-bbd0-e91dc17c1676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:17:25.976713 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68545296-a362-4da9-bbd0-e91dc17c1676 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:17:25.986824 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1165/2364] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:17:25.990892 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2389e7d-afe9-4f86-a9d7-4b2eec2fcfe1 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:17:26.001951 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2389e7d-afe9-4f86-a9d7-4b2eec2fcfe1 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:17:26.006699 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ef006-7673-4fb0-b3ab-3bdbc9b22656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:17:26.007287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ef006-7673-4fb0-b3ab-3bdbc9b22656 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:26.007567 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ef006-7673-4fb0-b3ab-3bdbc9b22656 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:26.007838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ef006-7673-4fb0-b3ab-3bdbc9b22656 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:26.008506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ef006-7673-4fb0-b3ab-3bdbc9b22656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:17:26.010094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ef006-7673-4fb0-b3ab-3bdbc9b22656 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:17:26.021425 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1200/2365] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:17:26.025703 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68545296-a362-4da9-bbd0-e91dc17c1676 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:17:26.036096 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68545296-a362-4da9-bbd0-e91dc17c1676 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:17:26.040798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84559f3f-09e0-40bb-81cf-5a1c12cd153e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:17:26.041433 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84559f3f-09e0-40bb-81cf-5a1c12cd153e admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:26.041709 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84559f3f-09e0-40bb-81cf-5a1c12cd153e admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:26.041975 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84559f3f-09e0-40bb-81cf-5a1c12cd153e admin admin] PATH_INFO: `/v3/projects/358470e28efc4cfea09fa536a8e9f113/users/eb2357b9a0ef46019960e102ed3727c2/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:26.056867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84559f3f-09e0-40bb-81cf-5a1c12cd153e admin admin] RBAC: Authorizing `identity:create_grant(project_id=358470e28efc4cfea09fa536a8e9f113, user_id=eb2357b9a0ef46019960e102ed3727c2, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:17:26.060385 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84559f3f-09e0-40bb-81cf-5a1c12cd153e admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:17:26.145174 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1166/2366] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:17:26 2022] PUT /identity/v3/projects/358470e28efc4cfea09fa536a8e9f113/users/eb2357b9a0ef46019960e102ed3727c2/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:17:26.158453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed8dbe-f220-4f06-9ca8-0a9876897958 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:26.158732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed8dbe-f220-4f06-9ca8-0a9876897958 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:26.158997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed8dbe-f220-4f06-9ca8-0a9876897958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:26.187571 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-54ed8dbe-f220-4f06-9ca8-0a9876897958 None None] MFA Rules not processed for user `eb2357b9a0ef46019960e102ed3727c2`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:17:26.188871 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-54ed8dbe-f220-4f06-9ca8-0a9876897958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:17:26.218528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:17:26.253437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1201/2367] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:17:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:17:26.258014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84559f3f-09e0-40bb-81cf-5a1c12cd153e admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:17:26.320601 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84559f3f-09e0-40bb-81cf-5a1c12cd153e admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:17:26.325942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fc43c-ff14-4c0a-92c4-18e07f18ae27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:17:26.326533 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fc43c-ff14-4c0a-92c4-18e07f18ae27 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:26.326807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fc43c-ff14-4c0a-92c4-18e07f18ae27 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:26.327073 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fc43c-ff14-4c0a-92c4-18e07f18ae27 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:26.327686 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479fc43c-ff14-4c0a-92c4-18e07f18ae27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:17:26.329642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479fc43c-ff14-4c0a-92c4-18e07f18ae27 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:17:26.438130 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1167/2368] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:17:26 2022] POST /identity/v3/projects => generated 369 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:17:26.442186 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed8dbe-f220-4f06-9ca8-0a9876897958 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:17:26.454479 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed8dbe-f220-4f06-9ca8-0a9876897958 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:17:26.509935 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cce3b-f8ad-498a-aacf-e021a6bda0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:17:26.510544 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cce3b-f8ad-498a-aacf-e021a6bda0de admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:26.510819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cce3b-f8ad-498a-aacf-e021a6bda0de admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:26.511085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cce3b-f8ad-498a-aacf-e021a6bda0de admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:26.511706 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29cce3b-f8ad-498a-aacf-e021a6bda0de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:17:26.514254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29cce3b-f8ad-498a-aacf-e021a6bda0de admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:17:26.613468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1202/2369] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:17:26 2022] POST /identity/v3/users => generated 291 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:17:26.617459 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fc43c-ff14-4c0a-92c4-18e07f18ae27 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:17:26.628192 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fc43c-ff14-4c0a-92c4-18e07f18ae27 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:17:26.633131 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631a86f-0501-479a-a262-2f76a60357b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:17:26.633740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631a86f-0501-479a-a262-2f76a60357b4 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:26.634020 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631a86f-0501-479a-a262-2f76a60357b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:26.634349 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631a86f-0501-479a-a262-2f76a60357b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:26.634991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631a86f-0501-479a-a262-2f76a60357b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:17:26.636689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631a86f-0501-479a-a262-2f76a60357b4 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:17:26.649141 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1168/2370] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:17:26.654270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cce3b-f8ad-498a-aacf-e021a6bda0de admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:17:26.664825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cce3b-f8ad-498a-aacf-e021a6bda0de admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:17:26.669983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcefdf-2d2d-4576-93aa-2d85a507655e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:17:26.670633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcefdf-2d2d-4576-93aa-2d85a507655e admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:26.670913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcefdf-2d2d-4576-93aa-2d85a507655e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:26.671190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcefdf-2d2d-4576-93aa-2d85a507655e admin admin] PATH_INFO: `/v3/projects/7e6000e0aaa54eed91adbe010039f8f6/users/393995c7298848ac9bb2467d7edaf96b/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:26.688919 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcefdf-2d2d-4576-93aa-2d85a507655e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e6000e0aaa54eed91adbe010039f8f6, user_id=393995c7298848ac9bb2467d7edaf96b, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:17:26.691690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcefdf-2d2d-4576-93aa-2d85a507655e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:17:26.762819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1203/2371] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:17:26 2022] PUT /identity/v3/projects/7e6000e0aaa54eed91adbe010039f8f6/users/393995c7298848ac9bb2467d7edaf96b/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:17:26.766948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631a86f-0501-479a-a262-2f76a60357b4 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:17:26.778795 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631a86f-0501-479a-a262-2f76a60357b4 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:17:26.788581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:17:26.836698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c683ec-032d-4971-87df-6bb055fa73f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:17:26.837301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c683ec-032d-4971-87df-6bb055fa73f9 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:26.837582 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c683ec-032d-4971-87df-6bb055fa73f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:26.837851 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c683ec-032d-4971-87df-6bb055fa73f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:26.838480 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c683ec-032d-4971-87df-6bb055fa73f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:17:26.840056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c683ec-032d-4971-87df-6bb055fa73f9 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:17:26.850695 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1169/2372] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:17:26.858759 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcefdf-2d2d-4576-93aa-2d85a507655e admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:17:26.866049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcefdf-2d2d-4576-93aa-2d85a507655e admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:17:26.870724 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9214ef-9149-4f94-b181-44be85480203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:17:26.871366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9214ef-9149-4f94-b181-44be85480203 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:26.871642 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9214ef-9149-4f94-b181-44be85480203 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:26.871909 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9214ef-9149-4f94-b181-44be85480203 admin admin] PATH_INFO: `/v3/projects/7e6000e0aaa54eed91adbe010039f8f6/users/393995c7298848ac9bb2467d7edaf96b/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:26.874342 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9214ef-9149-4f94-b181-44be85480203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e6000e0aaa54eed91adbe010039f8f6, user_id=393995c7298848ac9bb2467d7edaf96b, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:17:26.876987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9214ef-9149-4f94-b181-44be85480203 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:17:26.888102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1204/2373] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:17:26 2022] PUT /identity/v3/projects/7e6000e0aaa54eed91adbe010039f8f6/users/393995c7298848ac9bb2467d7edaf96b/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:17:26.899595 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a67c1f-35df-483a-9fed-a560b8a1b45d None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:17:26.899875 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a67c1f-35df-483a-9fed-a560b8a1b45d None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:17:26.900160 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a67c1f-35df-483a-9fed-a560b8a1b45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:17:26.929993 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-e1a67c1f-35df-483a-9fed-a560b8a1b45d None None] MFA Rules not processed for user `393995c7298848ac9bb2467d7edaf96b`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:17:26.931248 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-e1a67c1f-35df-483a-9fed-a560b8a1b45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:17:27.002138 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1170/2374] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:17:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:17:36.884321 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=57,keystone:INSERT=5 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:17:36.998658 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=57 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:18:21.783168 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9214ef-9149-4f94-b181-44be85480203 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:18:21.785218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6c9214ef-9149-4f94-b181-44be85480203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:18:21.855810 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9214ef-9149-4f94-b181-44be85480203 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:18:21.861430 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9688d-7b58-40e0-a373-28b0acac5ad6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:18:21.863890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9688d-7b58-40e0-a373-28b0acac5ad6 service placement] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:21.863890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9688d-7b58-40e0-a373-28b0acac5ad6 service placement] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:21.863890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9688d-7b58-40e0-a373-28b0acac5ad6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:21.864272 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8fc9688d-7b58-40e0-a373-28b0acac5ad6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:18:21.875557 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9688d-7b58-40e0-a373-28b0acac5ad6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:18:21.877948 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9688d-7b58-40e0-a373-28b0acac5ad6 service placement] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:18:21.887768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1205/2375] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:18:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:18:27.063215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a67c1f-35df-483a-9fed-a560b8a1b45d None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:18:27.126378 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a67c1f-35df-483a-9fed-a560b8a1b45d None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:18:27.130891 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c27a3-3211-4e34-acd1-8e2c09429058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:18:27.131519 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c27a3-3211-4e34-acd1-8e2c09429058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:27.131798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c27a3-3211-4e34-acd1-8e2c09429058 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:27.132071 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c27a3-3211-4e34-acd1-8e2c09429058 admin admin] PATH_INFO: `/v3/users/eb2357b9a0ef46019960e102ed3727c2` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:27.133247 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c27a3-3211-4e34-acd1-8e2c09429058 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb2357b9a0ef46019960e102ed3727c2)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:18:27.135156 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c27a3-3211-4e34-acd1-8e2c09429058 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:18:27.310965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-590c27a3-3211-4e34-acd1-8e2c09429058 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb2357b9a0ef46019960e102ed3727c2'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:18:27.318546 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-590c27a3-3211-4e34-acd1-8e2c09429058 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb2357b9a0ef46019960e102ed3727c2'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:18:27.319280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-590c27a3-3211-4e34-acd1-8e2c09429058 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb2357b9a0ef46019960e102ed3727c2'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:18:27.332452 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-590c27a3-3211-4e34-acd1-8e2c09429058 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb2357b9a0ef46019960e102ed3727c2'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:18:27.333207 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-590c27a3-3211-4e34-acd1-8e2c09429058 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb2357b9a0ef46019960e102ed3727c2'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:18:27.431278 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1171/2376] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:18:27 2022] DELETE /identity/v3/users/eb2357b9a0ef46019960e102ed3727c2 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:18:27.436121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d06405-a607-460e-8bd4-d1511594f4ba None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:27.436677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d06405-a607-460e-8bd4-d1511594f4ba None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:27.436677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d06405-a607-460e-8bd4-d1511594f4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:27.459952 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-36d06405-a607-460e-8bd4-d1511594f4ba None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:18:27.461432 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-36d06405-a607-460e-8bd4-d1511594f4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:18:27.524728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1206/2377] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:18:27.535618 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c27a3-3211-4e34-acd1-8e2c09429058 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:18:27.537221 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-590c27a3-3211-4e34-acd1-8e2c09429058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:18:27.602392 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c27a3-3211-4e34-acd1-8e2c09429058 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:18:27.608200 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a748144-da7b-43d4-ab3b-7068137ff1e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:18:27.608776 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a748144-da7b-43d4-ab3b-7068137ff1e1 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:27.609215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a748144-da7b-43d4-ab3b-7068137ff1e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:27.609215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a748144-da7b-43d4-ab3b-7068137ff1e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:27.620984 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a748144-da7b-43d4-ab3b-7068137ff1e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:18:27.622716 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a748144-da7b-43d4-ab3b-7068137ff1e1 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:18:27.634734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1172/2378] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:18:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:18:28.207210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d06405-a607-460e-8bd4-d1511594f4ba None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:18:28.208754 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-36d06405-a607-460e-8bd4-d1511594f4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:18:28.224384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d06405-a607-460e-8bd4-d1511594f4ba None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:18:28.226202 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65243fe-61a3-48ad-abc6-0d8a032aff1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:18:28.226878 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65243fe-61a3-48ad-abc6-0d8a032aff1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:28.227154 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65243fe-61a3-48ad-abc6-0d8a032aff1e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:28.227436 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65243fe-61a3-48ad-abc6-0d8a032aff1e admin admin] PATH_INFO: `/v3/projects/358470e28efc4cfea09fa536a8e9f113` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:28.228584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65243fe-61a3-48ad-abc6-0d8a032aff1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=358470e28efc4cfea09fa536a8e9f113)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:18:28.230475 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65243fe-61a3-48ad-abc6-0d8a032aff1e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:18:28.338693 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b65243fe-61a3-48ad-abc6-0d8a032aff1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '358470e28efc4cfea09fa536a8e9f113', 'request_id': 'req-b65243fe-61a3-48ad-abc6-0d8a032aff1e'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:18:28.345737 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b65243fe-61a3-48ad-abc6-0d8a032aff1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '358470e28efc4cfea09fa536a8e9f113', 'request_id': 'req-b65243fe-61a3-48ad-abc6-0d8a032aff1e'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:18:28.351512 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b65243fe-61a3-48ad-abc6-0d8a032aff1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '358470e28efc4cfea09fa536a8e9f113', 'request_id': 'req-b65243fe-61a3-48ad-abc6-0d8a032aff1e'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:18:28.466761 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b65243fe-61a3-48ad-abc6-0d8a032aff1e admin admin] The token cache is being invalidate because project 358470e28efc4cfea09fa536a8e9f113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:18:28.467578 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b65243fe-61a3-48ad-abc6-0d8a032aff1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:18:28.467775 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b65243fe-61a3-48ad-abc6-0d8a032aff1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:18:28.469184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1207/2379] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:18:28 2022] DELETE /identity/v3/projects/358470e28efc4cfea09fa536a8e9f113 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:18:28.474015 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a748144-da7b-43d4-ab3b-7068137ff1e1 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:18:28.484332 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a748144-da7b-43d4-ab3b-7068137ff1e1 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:18:28.505134 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:18:28.544849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3d60a-6115-49e9-921f-e4cb6ac44ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:18:28.545482 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3d60a-6115-49e9-921f-e4cb6ac44ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:28.545756 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3d60a-6115-49e9-921f-e4cb6ac44ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:28.546024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3d60a-6115-49e9-921f-e4cb6ac44ec4 admin admin] PATH_INFO: `/v3/users/393995c7298848ac9bb2467d7edaf96b` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:28.547133 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b3d60a-6115-49e9-921f-e4cb6ac44ec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=393995c7298848ac9bb2467d7edaf96b)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:18:28.549141 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b3d60a-6115-49e9-921f-e4cb6ac44ec4 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:18:28.768072 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-79b3d60a-6115-49e9-921f-e4cb6ac44ec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '393995c7298848ac9bb2467d7edaf96b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:18:28.775363 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-79b3d60a-6115-49e9-921f-e4cb6ac44ec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '393995c7298848ac9bb2467d7edaf96b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:18:28.776185 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-79b3d60a-6115-49e9-921f-e4cb6ac44ec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '393995c7298848ac9bb2467d7edaf96b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:18:28.789120 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-79b3d60a-6115-49e9-921f-e4cb6ac44ec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '393995c7298848ac9bb2467d7edaf96b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:18:28.789769 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-79b3d60a-6115-49e9-921f-e4cb6ac44ec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '393995c7298848ac9bb2467d7edaf96b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:18:28.851529 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1173/2380] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:18:28 2022] DELETE /identity/v3/users/393995c7298848ac9bb2467d7edaf96b => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:18:29.263958 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65243fe-61a3-48ad-abc6-0d8a032aff1e admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:18:29.265415 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b65243fe-61a3-48ad-abc6-0d8a032aff1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:18:29.280958 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65243fe-61a3-48ad-abc6-0d8a032aff1e admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:18:29.290955 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:18:29.338854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ae95b-bfdd-4c49-953c-749de0401863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:18:29.339487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ae95b-bfdd-4c49-953c-749de0401863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:29.339763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ae95b-bfdd-4c49-953c-749de0401863 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:29.340033 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ae95b-bfdd-4c49-953c-749de0401863 admin admin] PATH_INFO: `/v3/projects/7e6000e0aaa54eed91adbe010039f8f6` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:29.341223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ae95b-bfdd-4c49-953c-749de0401863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e6000e0aaa54eed91adbe010039f8f6)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:18:29.343615 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ae95b-bfdd-4c49-953c-749de0401863 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:18:29.447226 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-373ae95b-bfdd-4c49-953c-749de0401863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e6000e0aaa54eed91adbe010039f8f6', 'request_id': 'req-373ae95b-bfdd-4c49-953c-749de0401863'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:18:29.453645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-373ae95b-bfdd-4c49-953c-749de0401863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e6000e0aaa54eed91adbe010039f8f6', 'request_id': 'req-373ae95b-bfdd-4c49-953c-749de0401863'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:18:29.459395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-373ae95b-bfdd-4c49-953c-749de0401863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e6000e0aaa54eed91adbe010039f8f6', 'request_id': 'req-373ae95b-bfdd-4c49-953c-749de0401863'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:18:29.537208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-373ae95b-bfdd-4c49-953c-749de0401863 admin admin] The token cache is being invalidate because project 7e6000e0aaa54eed91adbe010039f8f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:18:29.537854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-373ae95b-bfdd-4c49-953c-749de0401863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:18:29.538091 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-373ae95b-bfdd-4c49-953c-749de0401863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:18:29.539595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1208/2381] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:18:29 2022] DELETE /identity/v3/projects/7e6000e0aaa54eed91adbe010039f8f6 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:18:29.547707 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef36f4f-5218-4ef9-828d-946832aa534d None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:29.552088 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef36f4f-5218-4ef9-828d-946832aa534d None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:29.552088 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef36f4f-5218-4ef9-828d-946832aa534d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:29.569148 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-fef36f4f-5218-4ef9-828d-946832aa534d None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:18:29.570473 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-fef36f4f-5218-4ef9-828d-946832aa534d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:18:29.630533 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1174/2382] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:18:29.638185 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236cd11-c313-47d8-b7b1-a0cfe89c9b40 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:29.638465 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236cd11-c313-47d8-b7b1-a0cfe89c9b40 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:29.638740 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236cd11-c313-47d8-b7b1-a0cfe89c9b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:29.659304 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-7236cd11-c313-47d8-b7b1-a0cfe89c9b40 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:18:29.660546 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-7236cd11-c313-47d8-b7b1-a0cfe89c9b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:18:29.668558 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1209/2383] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:18:29.672728 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef36f4f-5218-4ef9-828d-946832aa534d None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:18:29.683020 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef36f4f-5218-4ef9-828d-946832aa534d None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:18:29.687815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ef6d8-7c02-46bf-b8ad-2e2bff922487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:18:29.688475 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ef6d8-7c02-46bf-b8ad-2e2bff922487 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:29.688836 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ef6d8-7c02-46bf-b8ad-2e2bff922487 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:29.689073 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ef6d8-7c02-46bf-b8ad-2e2bff922487 admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:29.689889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ef6d8-7c02-46bf-b8ad-2e2bff922487 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:18:29.690122 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ef6d8-7c02-46bf-b8ad-2e2bff922487 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:18:29.691773 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ef6d8-7c02-46bf-b8ad-2e2bff922487 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:18:29.703260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1175/2384] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:18:29.707855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236cd11-c313-47d8-b7b1-a0cfe89c9b40 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:18:29.717245 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236cd11-c313-47d8-b7b1-a0cfe89c9b40 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:18:29.722618 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae999bf-f211-4b62-bc57-7d2dd2227ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:18:29.723207 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae999bf-f211-4b62-bc57-7d2dd2227ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:29.723485 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae999bf-f211-4b62-bc57-7d2dd2227ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:29.723755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae999bf-f211-4b62-bc57-7d2dd2227ab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:29.724412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae999bf-f211-4b62-bc57-7d2dd2227ab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:18:29.726318 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae999bf-f211-4b62-bc57-7d2dd2227ab8 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:18:29.836110 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1210/2385] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:18:29 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:18:29.840187 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ef6d8-7c02-46bf-b8ad-2e2bff922487 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:18:29.850905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ef6d8-7c02-46bf-b8ad-2e2bff922487 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:18:29.883892 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:18:29.916655 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b717d23-86ef-4bd1-8a65-449e1eb00e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:18:29.917249 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b717d23-86ef-4bd1-8a65-449e1eb00e16 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:29.917527 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b717d23-86ef-4bd1-8a65-449e1eb00e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:29.917803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b717d23-86ef-4bd1-8a65-449e1eb00e16 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:29.918429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b717d23-86ef-4bd1-8a65-449e1eb00e16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:18:29.920354 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b717d23-86ef-4bd1-8a65-449e1eb00e16 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:18:30.029403 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1176/2386] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:18:29 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:18:30.035172 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae999bf-f211-4b62-bc57-7d2dd2227ab8 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:18:30.050968 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae999bf-f211-4b62-bc57-7d2dd2227ab8 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:18:30.056037 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18017c-f144-409b-b9f8-0cfebb9a0e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:18:30.056742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18017c-f144-409b-b9f8-0cfebb9a0e13 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:30.057014 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18017c-f144-409b-b9f8-0cfebb9a0e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:30.057313 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18017c-f144-409b-b9f8-0cfebb9a0e13 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:30.058016 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b18017c-f144-409b-b9f8-0cfebb9a0e13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:18:30.059652 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b18017c-f144-409b-b9f8-0cfebb9a0e13 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:18:30.071048 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1211/2387] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:18:30.076449 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b717d23-86ef-4bd1-8a65-449e1eb00e16 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:18:30.087770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b717d23-86ef-4bd1-8a65-449e1eb00e16 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:18:30.092592 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06be5f-dc5d-4b95-900c-138c1ea62fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:18:30.093200 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06be5f-dc5d-4b95-900c-138c1ea62fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:30.093476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06be5f-dc5d-4b95-900c-138c1ea62fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:30.093750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06be5f-dc5d-4b95-900c-138c1ea62fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:30.094395 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb06be5f-dc5d-4b95-900c-138c1ea62fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:18:30.096718 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb06be5f-dc5d-4b95-900c-138c1ea62fe8 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:18:30.107934 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1177/2388] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:18:30.112599 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18017c-f144-409b-b9f8-0cfebb9a0e13 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:18:30.122713 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18017c-f144-409b-b9f8-0cfebb9a0e13 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:18:30.127963 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713567a-40cd-4fd7-be58-1a02f3b74794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:18:30.128755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713567a-40cd-4fd7-be58-1a02f3b74794 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:30.129012 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713567a-40cd-4fd7-be58-1a02f3b74794 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:30.129281 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713567a-40cd-4fd7-be58-1a02f3b74794 admin admin] PATH_INFO: `/v3/projects/0b1cc9acbc3a4109bb306e66b01c2659/users/3e2b0c2a68704a429f11aec9d124b1b9/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:30.145529 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f713567a-40cd-4fd7-be58-1a02f3b74794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b1cc9acbc3a4109bb306e66b01c2659, user_id=3e2b0c2a68704a429f11aec9d124b1b9, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:18:30.148411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f713567a-40cd-4fd7-be58-1a02f3b74794 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:18:30.252753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1212/2389] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:18:30 2022] PUT /identity/v3/projects/0b1cc9acbc3a4109bb306e66b01c2659/users/3e2b0c2a68704a429f11aec9d124b1b9/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:18:30.320844 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0152e-d8f9-49ff-9fb3-9d9f054d193e None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:30.321159 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0152e-d8f9-49ff-9fb3-9d9f054d193e None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:30.321385 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0152e-d8f9-49ff-9fb3-9d9f054d193e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:30.350283 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-96b0152e-d8f9-49ff-9fb3-9d9f054d193e None None] MFA Rules not processed for user `3e2b0c2a68704a429f11aec9d124b1b9`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:18:30.351412 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-96b0152e-d8f9-49ff-9fb3-9d9f054d193e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:18:30.421998 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1178/2390] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:18:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:18:30.427194 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713567a-40cd-4fd7-be58-1a02f3b74794 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:18:30.469642 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:18:30.490745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713567a-40cd-4fd7-be58-1a02f3b74794 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:18:30.495748 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc9e0e-12b4-4d95-b458-28c948b43984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:18:30.496687 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc9e0e-12b4-4d95-b458-28c948b43984 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:30.497184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc9e0e-12b4-4d95-b458-28c948b43984 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:30.497472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc9e0e-12b4-4d95-b458-28c948b43984 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:30.498606 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc9e0e-12b4-4d95-b458-28c948b43984 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:18:30.500644 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc9e0e-12b4-4d95-b458-28c948b43984 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:18:30.596668 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1213/2391] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:18:30 2022] POST /identity/v3/projects => generated 387 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:18:30.601489 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0152e-d8f9-49ff-9fb3-9d9f054d193e None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:18:30.615542 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0152e-d8f9-49ff-9fb3-9d9f054d193e None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:18:30.674767 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0ca53-728c-4f79-b4e6-e767ef7e6859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:18:30.675368 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0ca53-728c-4f79-b4e6-e767ef7e6859 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:30.675637 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0ca53-728c-4f79-b4e6-e767ef7e6859 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:30.675901 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0ca53-728c-4f79-b4e6-e767ef7e6859 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:30.676568 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0ca53-728c-4f79-b4e6-e767ef7e6859 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:18:30.678487 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0ca53-728c-4f79-b4e6-e767ef7e6859 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:18:30.748728 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1179/2392] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:18:30 2022] POST /identity/v3/users => generated 300 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:18:30.753089 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc9e0e-12b4-4d95-b458-28c948b43984 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:18:30.763681 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc9e0e-12b4-4d95-b458-28c948b43984 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:18:30.769375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe84bf-d7a2-4f0c-8bea-7fbae5197cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:18:30.770019 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe84bf-d7a2-4f0c-8bea-7fbae5197cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:30.770296 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe84bf-d7a2-4f0c-8bea-7fbae5197cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:30.770564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe84bf-d7a2-4f0c-8bea-7fbae5197cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:30.771350 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe84bf-d7a2-4f0c-8bea-7fbae5197cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:18:30.773529 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe84bf-d7a2-4f0c-8bea-7fbae5197cd7 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:18:30.785553 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1214/2393] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:18:30.790021 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0ca53-728c-4f79-b4e6-e767ef7e6859 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:18:30.800491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0ca53-728c-4f79-b4e6-e767ef7e6859 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:18:30.805265 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbfc56-dc68-40fe-a087-3e0e22d5ebf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:18:30.805920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbfc56-dc68-40fe-a087-3e0e22d5ebf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:30.806197 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbfc56-dc68-40fe-a087-3e0e22d5ebf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:30.807906 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbfc56-dc68-40fe-a087-3e0e22d5ebf1 admin admin] PATH_INFO: `/v3/projects/89e8253124fe4830ac08d7c24a05efbf/users/f7a2c495487a43b28e04702dc415ba6f/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:30.821894 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbfc56-dc68-40fe-a087-3e0e22d5ebf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89e8253124fe4830ac08d7c24a05efbf, user_id=f7a2c495487a43b28e04702dc415ba6f, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:18:30.824586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbfc56-dc68-40fe-a087-3e0e22d5ebf1 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:18:30.898044 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1180/2394] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:18:30 2022] PUT /identity/v3/projects/89e8253124fe4830ac08d7c24a05efbf/users/f7a2c495487a43b28e04702dc415ba6f/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:18:30.902200 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe84bf-d7a2-4f0c-8bea-7fbae5197cd7 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:18:30.913535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe84bf-d7a2-4f0c-8bea-7fbae5197cd7 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:18:30.972372 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c55323-ceb3-48ba-ad49-23ab944fb2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:18:30.973062 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c55323-ceb3-48ba-ad49-23ab944fb2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:30.973340 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c55323-ceb3-48ba-ad49-23ab944fb2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:30.973613 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c55323-ceb3-48ba-ad49-23ab944fb2c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:30.974252 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c55323-ceb3-48ba-ad49-23ab944fb2c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:18:30.975854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c55323-ceb3-48ba-ad49-23ab944fb2c0 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:18:30.986283 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1215/2395] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:18:30.991164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbfc56-dc68-40fe-a087-3e0e22d5ebf1 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:18:30.996866 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:18:31.003052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbfc56-dc68-40fe-a087-3e0e22d5ebf1 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:18:31.007894 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ad5c3-9db1-46f3-a128-12c1b3245e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:18:31.009686 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ad5c3-9db1-46f3-a128-12c1b3245e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:31.009686 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ad5c3-9db1-46f3-a128-12c1b3245e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:31.009686 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ad5c3-9db1-46f3-a128-12c1b3245e63 admin admin] PATH_INFO: `/v3/projects/89e8253124fe4830ac08d7c24a05efbf/users/f7a2c495487a43b28e04702dc415ba6f/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:31.011049 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ad5c3-9db1-46f3-a128-12c1b3245e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89e8253124fe4830ac08d7c24a05efbf, user_id=f7a2c495487a43b28e04702dc415ba6f, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:18:31.013734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ad5c3-9db1-46f3-a128-12c1b3245e63 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:18:31.024878 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1181/2396] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:18:30 2022] PUT /identity/v3/projects/89e8253124fe4830ac08d7c24a05efbf/users/f7a2c495487a43b28e04702dc415ba6f/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:18:31.035890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7852a3-62a8-46b6-b28d-2a7ffaebc6b2 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:18:31.038429 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7852a3-62a8-46b6-b28d-2a7ffaebc6b2 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:18:31.038429 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7852a3-62a8-46b6-b28d-2a7ffaebc6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:18:31.065109 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-ea7852a3-62a8-46b6-b28d-2a7ffaebc6b2 None None] MFA Rules not processed for user `f7a2c495487a43b28e04702dc415ba6f`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:18:31.066336 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-ea7852a3-62a8-46b6-b28d-2a7ffaebc6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:18:31.136060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1216/2397] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:18:31 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:18:41.056597 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=4,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:18:41.133077 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=83,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:11.971709 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ad5c3-9db1-46f3-a128-12c1b3245e63 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:11.973495 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d25ad5c3-9db1-46f3-a128-12c1b3245e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:12.014796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7852a3-62a8-46b6-b28d-2a7ffaebc6b2 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:12.016246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-ea7852a3-62a8-46b6-b28d-2a7ffaebc6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:12.043401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ad5c3-9db1-46f3-a128-12c1b3245e63 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:12.047280 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf6826-5ed2-4a1f-b0da-08b439cb4a72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:12.047888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf6826-5ed2-4a1f-b0da-08b439cb4a72 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:12.048190 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf6826-5ed2-4a1f-b0da-08b439cb4a72 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:12.048470 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf6826-5ed2-4a1f-b0da-08b439cb4a72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:12.050332 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-92bf6826-5ed2-4a1f-b0da-08b439cb4a72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:12.082265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7852a3-62a8-46b6-b28d-2a7ffaebc6b2 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:12.086121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0fae5-6f71-4af9-9aff-67dc8e79c66c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:12.089163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0fae5-6f71-4af9-9aff-67dc8e79c66c service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:12.089163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0fae5-6f71-4af9-9aff-67dc8e79c66c service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:12.089163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0fae5-6f71-4af9-9aff-67dc8e79c66c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:12.089163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-00a0fae5-6f71-4af9-9aff-67dc8e79c66c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:12.102639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application [None req-00a0fae5-6f71-4af9-9aff-67dc8e79c66c service cinder] Could not find project: 7e6000e0aaa54eed91adbe010039f8f6.: keystone.exception.ProjectNotFound: Could not find project: 7e6000e0aaa54eed91adbe010039f8f6. Aug 29 23:19:12.102639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 23:19:12.102639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 23:19:12.102639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 23:19:12.102639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 23:19:12.102639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 23:19:12.102639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 23:19:12.102639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 23:19:12.102639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 23:19:12.102639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 23:19:12.102639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 23:19:12.102639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 23:19:12.102639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Aug 29 23:19:12.102639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Aug 29 23:19:12.102639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Aug 29 23:19:12.104158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Aug 29 23:19:12.104158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Aug 29 23:19:12.104158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Aug 29 23:19:12.104158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:19:12.104158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:19:12.104158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Aug 29 23:19:12.104158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Aug 29 23:19:12.104158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:19:12.104158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:19:12.104158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:19:12.104158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:19:12.104158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:19:12.104158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 23:19:12.104158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:19:12.104158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:19:12.104158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:19:12.104158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:19:12.105489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:19:12.105489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:19:12.105489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:19:12.105489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:19:12.105489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Aug 29 23:19:12.105489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Aug 29 23:19:12.105489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Aug 29 23:19:12.105489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self._validate_token_resources() Aug 29 23:19:12.105489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Aug 29 23:19:12.105489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Aug 29 23:19:12.105489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Aug 29 23:19:12.105489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Aug 29 23:19:12.105489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 23:19:12.105489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 23:19:12.105489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 23:19:12.105489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 23:19:12.105489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 23:19:12.107032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 23:19:12.107032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 23:19:12.107032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application with Lock( Aug 29 23:19:12.107032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 23:19:12.107032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._enter() Aug 29 23:19:12.107032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 23:19:12.107032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 23:19:12.107032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 23:19:12.107032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.creator() Aug 29 23:19:12.107032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 23:19:12.107032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application created_value = creator( Aug 29 23:19:12.107032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 23:19:12.107032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 23:19:12.107032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 23:19:12.107032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 23:19:12.107032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 23:19:12.107032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 23:19:12.112484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 7e6000e0aaa54eed91adbe010039f8f6. Aug 29 23:19:12.112484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: ERROR keystone.server.flask.application Aug 29 23:19:12.112484 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1217/2398] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:12 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 90 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 29 23:19:12.142790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0fae5-6f71-4af9-9aff-67dc8e79c66c service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:12.144931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-00a0fae5-6f71-4af9-9aff-67dc8e79c66c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:12.150090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf6826-5ed2-4a1f-b0da-08b439cb4a72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:12.151572 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf6826-5ed2-4a1f-b0da-08b439cb4a72 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:12.175604 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1182/2399] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:19:12.238432 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0fae5-6f71-4af9-9aff-67dc8e79c66c service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:12.241918 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf6826-5ed2-4a1f-b0da-08b439cb4a72 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:12.249259 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f05dcb-bce8-4d7a-9530-19187de931d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:12.255814 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f05dcb-bce8-4d7a-9530-19187de931d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:12.255814 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f05dcb-bce8-4d7a-9530-19187de931d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:12.255814 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f05dcb-bce8-4d7a-9530-19187de931d3 admin admin] PATH_INFO: `/v3/users/2288d88eb89c4c798007acd9974c6bf0` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:12.255814 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f05dcb-bce8-4d7a-9530-19187de931d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2288d88eb89c4c798007acd9974c6bf0)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:12.255814 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f05dcb-bce8-4d7a-9530-19187de931d3 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:12.256593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf6826-5ed2-4a1f-b0da-08b439cb4a72 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:12.263735 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d4a7b-d140-4d20-bf47-b4ef4931f153 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:12.264388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d4a7b-d140-4d20-bf47-b4ef4931f153 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:12.264664 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d4a7b-d140-4d20-bf47-b4ef4931f153 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:12.264985 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d4a7b-d140-4d20-bf47-b4ef4931f153 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:12.276908 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d4a7b-d140-4d20-bf47-b4ef4931f153 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:12.278690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d4a7b-d140-4d20-bf47-b4ef4931f153 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:12.290676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1183/2400] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:19:12.325086 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d4a7b-d140-4d20-bf47-b4ef4931f153 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:12.342864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d4a7b-d140-4d20-bf47-b4ef4931f153 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:12.348126 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc01f44-1fa3-4ef4-9f58-cb1e8e433330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:12.349566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc01f44-1fa3-4ef4-9f58-cb1e8e433330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:12.349566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc01f44-1fa3-4ef4-9f58-cb1e8e433330 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:12.349883 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc01f44-1fa3-4ef4-9f58-cb1e8e433330 admin admin] PATH_INFO: `/v3/users/3e2b0c2a68704a429f11aec9d124b1b9` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:12.350994 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc01f44-1fa3-4ef4-9f58-cb1e8e433330 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e2b0c2a68704a429f11aec9d124b1b9)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:12.353891 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc01f44-1fa3-4ef4-9f58-cb1e8e433330 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:12.464579 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-f5f05dcb-bce8-4d7a-9530-19187de931d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2288d88eb89c4c798007acd9974c6bf0'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:12.465233 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-f5f05dcb-bce8-4d7a-9530-19187de931d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2288d88eb89c4c798007acd9974c6bf0'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:12.481713 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-f5f05dcb-bce8-4d7a-9530-19187de931d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2288d88eb89c4c798007acd9974c6bf0'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:12.553591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0bc01f44-1fa3-4ef4-9f58-cb1e8e433330 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e2b0c2a68704a429f11aec9d124b1b9'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:12.553942 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-f5f05dcb-bce8-4d7a-9530-19187de931d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2288d88eb89c4c798007acd9974c6bf0'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:12.554614 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-f5f05dcb-bce8-4d7a-9530-19187de931d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2288d88eb89c4c798007acd9974c6bf0'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:12.563122 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0bc01f44-1fa3-4ef4-9f58-cb1e8e433330 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e2b0c2a68704a429f11aec9d124b1b9'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:12.564412 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0bc01f44-1fa3-4ef4-9f58-cb1e8e433330 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e2b0c2a68704a429f11aec9d124b1b9'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:12.576225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:12.581907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0bc01f44-1fa3-4ef4-9f58-cb1e8e433330 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e2b0c2a68704a429f11aec9d124b1b9'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:12.584341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0bc01f44-1fa3-4ef4-9f58-cb1e8e433330 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e2b0c2a68704a429f11aec9d124b1b9'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:12.593511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1218/2401] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:19:12 2022] DELETE /identity/v3/users/2288d88eb89c4c798007acd9974c6bf0 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:19:12.600288 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905bafe-45b6-4c8d-9575-4d10b597d719 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:12.600288 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905bafe-45b6-4c8d-9575-4d10b597d719 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:12.600288 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905bafe-45b6-4c8d-9575-4d10b597d719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:12.619203 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-e905bafe-45b6-4c8d-9575-4d10b597d719 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:12.620479 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e905bafe-45b6-4c8d-9575-4d10b597d719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:12.642435 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:12.686803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1184/2402] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:19:12 2022] DELETE /identity/v3/users/3e2b0c2a68704a429f11aec9d124b1b9 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:12.695711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd143cb0-2b08-4b23-8b88-020da74d3f11 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:12.695711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd143cb0-2b08-4b23-8b88-020da74d3f11 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:12.695711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd143cb0-2b08-4b23-8b88-020da74d3f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:12.712249 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1219/2403] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:19:12.727191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905bafe-45b6-4c8d-9575-4d10b597d719 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:12.729273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e905bafe-45b6-4c8d-9575-4d10b597d719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:12.733741 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-cd143cb0-2b08-4b23-8b88-020da74d3f11 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:12.734972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-cd143cb0-2b08-4b23-8b88-020da74d3f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:12.815740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1185/2404] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:12.824989 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905bafe-45b6-4c8d-9575-4d10b597d719 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:12.834794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd143cb0-2b08-4b23-8b88-020da74d3f11 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:12.884668 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3a9b0-ee26-4ec1-a42d-7c68c6d7997f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:12.889567 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3a9b0-ee26-4ec1-a42d-7c68c6d7997f service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:12.889567 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3a9b0-ee26-4ec1-a42d-7c68c6d7997f service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:12.889567 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3a9b0-ee26-4ec1-a42d-7c68c6d7997f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:12.895471 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3a9b0-ee26-4ec1-a42d-7c68c6d7997f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:12.896983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3a9b0-ee26-4ec1-a42d-7c68c6d7997f service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:12.901218 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd143cb0-2b08-4b23-8b88-020da74d3f11 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:12.905052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec1dcee-6f7a-41b8-8389-a4753ef26750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:12.905718 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1dcee-6f7a-41b8-8389-a4753ef26750 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:12.905972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1dcee-6f7a-41b8-8389-a4753ef26750 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:12.906206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1dcee-6f7a-41b8-8389-a4753ef26750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:12.910128 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1220/2405] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:19:12.918319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec1dcee-6f7a-41b8-8389-a4753ef26750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:12.920940 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec1dcee-6f7a-41b8-8389-a4753ef26750 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:12.937084 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1186/2406] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:19:13.537322 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3a9b0-ee26-4ec1-a42d-7c68c6d7997f service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:13.540505 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec1dcee-6f7a-41b8-8389-a4753ef26750 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:13.541697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c1c3a9b0-ee26-4ec1-a42d-7c68c6d7997f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:13.542881 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-dec1dcee-6f7a-41b8-8389-a4753ef26750 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:13.568450 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec1dcee-6f7a-41b8-8389-a4753ef26750 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:13.570730 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3a9b0-ee26-4ec1-a42d-7c68c6d7997f service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:13.581407 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2eb672-35b5-4f80-921d-d99d393a6d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:13.583991 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2eb672-35b5-4f80-921d-d99d393a6d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:13.584324 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2eb672-35b5-4f80-921d-d99d393a6d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:13.584995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9968ca-82ce-4241-8550-51346d3af55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:13.585227 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2eb672-35b5-4f80-921d-d99d393a6d2b admin admin] PATH_INFO: `/v3/projects/0b1cc9acbc3a4109bb306e66b01c2659` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:13.585966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9968ca-82ce-4241-8550-51346d3af55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:13.586458 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9968ca-82ce-4241-8550-51346d3af55f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:13.586907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9968ca-82ce-4241-8550-51346d3af55f admin admin] PATH_INFO: `/v3/projects/4e4d6cd42d224f1d88ba9c7736727482` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:13.587455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2eb672-35b5-4f80-921d-d99d393a6d2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b1cc9acbc3a4109bb306e66b01c2659)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:13.589646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9968ca-82ce-4241-8550-51346d3af55f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e4d6cd42d224f1d88ba9c7736727482)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:13.591527 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2eb672-35b5-4f80-921d-d99d393a6d2b admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:13.592412 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9968ca-82ce-4241-8550-51346d3af55f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:13.737435 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-5e2eb672-35b5-4f80-921d-d99d393a6d2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b1cc9acbc3a4109bb306e66b01c2659', 'request_id': 'req-5e2eb672-35b5-4f80-921d-d99d393a6d2b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:13.745786 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-5e2eb672-35b5-4f80-921d-d99d393a6d2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b1cc9acbc3a4109bb306e66b01c2659', 'request_id': 'req-5e2eb672-35b5-4f80-921d-d99d393a6d2b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:13.749202 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0c9968ca-82ce-4241-8550-51346d3af55f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e4d6cd42d224f1d88ba9c7736727482', 'request_id': 'req-0c9968ca-82ce-4241-8550-51346d3af55f'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:13.751550 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-5e2eb672-35b5-4f80-921d-d99d393a6d2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b1cc9acbc3a4109bb306e66b01c2659', 'request_id': 'req-5e2eb672-35b5-4f80-921d-d99d393a6d2b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:13.827320 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0c9968ca-82ce-4241-8550-51346d3af55f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e4d6cd42d224f1d88ba9c7736727482', 'request_id': 'req-0c9968ca-82ce-4241-8550-51346d3af55f'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:13.833588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0c9968ca-82ce-4241-8550-51346d3af55f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e4d6cd42d224f1d88ba9c7736727482', 'request_id': 'req-0c9968ca-82ce-4241-8550-51346d3af55f'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:13.844739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0c9968ca-82ce-4241-8550-51346d3af55f admin admin] The token cache is being invalidate because project 4e4d6cd42d224f1d88ba9c7736727482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:13.845029 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0c9968ca-82ce-4241-8550-51346d3af55f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:13.845665 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0c9968ca-82ce-4241-8550-51346d3af55f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:13.847205 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1187/2407] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:13 2022] DELETE /identity/v3/projects/4e4d6cd42d224f1d88ba9c7736727482 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:13.847963 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9968ca-82ce-4241-8550-51346d3af55f admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:13.849283 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-0c9968ca-82ce-4241-8550-51346d3af55f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:13.862580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:13.899308 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-5e2eb672-35b5-4f80-921d-d99d393a6d2b admin admin] The token cache is being invalidate because project 0b1cc9acbc3a4109bb306e66b01c2659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:13.899601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-5e2eb672-35b5-4f80-921d-d99d393a6d2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:13.900288 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-5e2eb672-35b5-4f80-921d-d99d393a6d2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:13.901812 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1221/2408] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:13 2022] DELETE /identity/v3/projects/0b1cc9acbc3a4109bb306e66b01c2659 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:13.902572 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2eb672-35b5-4f80-921d-d99d393a6d2b admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:13.903854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-5e2eb672-35b5-4f80-921d-d99d393a6d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:13.912189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9968ca-82ce-4241-8550-51346d3af55f admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:13.918611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368cf6c-17f2-447b-bf59-14514a52daef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:13.918611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368cf6c-17f2-447b-bf59-14514a52daef service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:13.918611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368cf6c-17f2-447b-bf59-14514a52daef service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:13.918611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368cf6c-17f2-447b-bf59-14514a52daef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:13.919403 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-7368cf6c-17f2-447b-bf59-14514a52daef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:13.923153 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:13.970701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2eb672-35b5-4f80-921d-d99d393a6d2b admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:14.004899 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368cf6c-17f2-447b-bf59-14514a52daef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:14.006949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368cf6c-17f2-447b-bf59-14514a52daef service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:14.028478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1188/2409] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:19:14.029548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368cf6c-17f2-447b-bf59-14514a52daef service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:14.030965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-7368cf6c-17f2-447b-bf59-14514a52daef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:14.031482 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d3cd4-dfc8-432d-bea2-a85a5a63ac4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:14.032396 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d3cd4-dfc8-432d-bea2-a85a5a63ac4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:14.032669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d3cd4-dfc8-432d-bea2-a85a5a63ac4f admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:14.032934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d3cd4-dfc8-432d-bea2-a85a5a63ac4f admin admin] PATH_INFO: `/v3/users/2a86212007c044f6b3feade9646d0038` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:14.034064 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d3cd4-dfc8-432d-bea2-a85a5a63ac4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a86212007c044f6b3feade9646d0038)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:14.036026 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d3cd4-dfc8-432d-bea2-a85a5a63ac4f admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:14.044836 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368cf6c-17f2-447b-bf59-14514a52daef service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:14.049564 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1a2f8-7048-493a-9137-256964170f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:14.050221 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1a2f8-7048-493a-9137-256964170f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:14.050494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1a2f8-7048-493a-9137-256964170f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:14.050769 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1a2f8-7048-493a-9137-256964170f4c admin admin] PATH_INFO: `/v3/users/f7a2c495487a43b28e04702dc415ba6f` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:14.052496 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1a2f8-7048-493a-9137-256964170f4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7a2c495487a43b28e04702dc415ba6f)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:14.054836 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1a2f8-7048-493a-9137-256964170f4c admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:14.264554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-39d1a2f8-7048-493a-9137-256964170f4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7a2c495487a43b28e04702dc415ba6f'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:14.264874 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6f5d3cd4-dfc8-432d-bea2-a85a5a63ac4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a86212007c044f6b3feade9646d0038'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:14.265346 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6f5d3cd4-dfc8-432d-bea2-a85a5a63ac4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a86212007c044f6b3feade9646d0038'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:14.271766 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-39d1a2f8-7048-493a-9137-256964170f4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7a2c495487a43b28e04702dc415ba6f'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:14.272466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-39d1a2f8-7048-493a-9137-256964170f4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7a2c495487a43b28e04702dc415ba6f'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:14.273152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6f5d3cd4-dfc8-432d-bea2-a85a5a63ac4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a86212007c044f6b3feade9646d0038'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:14.287607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-39d1a2f8-7048-493a-9137-256964170f4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7a2c495487a43b28e04702dc415ba6f'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:14.289512 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-39d1a2f8-7048-493a-9137-256964170f4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7a2c495487a43b28e04702dc415ba6f'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:14.329758 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6f5d3cd4-dfc8-432d-bea2-a85a5a63ac4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a86212007c044f6b3feade9646d0038'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:14.330393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6f5d3cd4-dfc8-432d-bea2-a85a5a63ac4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a86212007c044f6b3feade9646d0038'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:14.348376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1222/2410] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:19:13 2022] DELETE /identity/v3/users/2a86212007c044f6b3feade9646d0038 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:14.404167 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1189/2411] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:19:14 2022] DELETE /identity/v3/users/f7a2c495487a43b28e04702dc415ba6f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:14.822359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d3cd4-dfc8-432d-bea2-a85a5a63ac4f admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:14.822359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6f5d3cd4-dfc8-432d-bea2-a85a5a63ac4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:14.869140 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:14.887588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1a2f8-7048-493a-9137-256964170f4c admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:14.889198 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-39d1a2f8-7048-493a-9137-256964170f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:14.901802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:14.905574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d3cd4-dfc8-432d-bea2-a85a5a63ac4f admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:14.965548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1a2f8-7048-493a-9137-256964170f4c admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:14.971765 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d444e4-1ebd-44fb-b421-68d56ad9aa0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:14.975700 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d444e4-1ebd-44fb-b421-68d56ad9aa0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:14.975700 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d444e4-1ebd-44fb-b421-68d56ad9aa0f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:14.975700 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d444e4-1ebd-44fb-b421-68d56ad9aa0f admin admin] PATH_INFO: `/v3/projects/89e8253124fe4830ac08d7c24a05efbf` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:14.976295 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f7794-bf7b-48d3-80b7-ebad3b90b6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:14.976427 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d444e4-1ebd-44fb-b421-68d56ad9aa0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=89e8253124fe4830ac08d7c24a05efbf)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:14.976536 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f7794-bf7b-48d3-80b7-ebad3b90b6c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:14.976536 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f7794-bf7b-48d3-80b7-ebad3b90b6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:14.976536 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f7794-bf7b-48d3-80b7-ebad3b90b6c3 admin admin] PATH_INFO: `/v3/projects/96732f1eeaf64fc8a23e8478c0c2f4c6` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:14.976812 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d444e4-1ebd-44fb-b421-68d56ad9aa0f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:14.977659 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f7794-bf7b-48d3-80b7-ebad3b90b6c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96732f1eeaf64fc8a23e8478c0c2f4c6)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:14.979806 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f7794-bf7b-48d3-80b7-ebad3b90b6c3 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:15.124721 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-cc3f7794-bf7b-48d3-80b7-ebad3b90b6c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96732f1eeaf64fc8a23e8478c0c2f4c6', 'request_id': 'req-cc3f7794-bf7b-48d3-80b7-ebad3b90b6c3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:15.131595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-cc3f7794-bf7b-48d3-80b7-ebad3b90b6c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96732f1eeaf64fc8a23e8478c0c2f4c6', 'request_id': 'req-cc3f7794-bf7b-48d3-80b7-ebad3b90b6c3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:15.137422 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-cc3f7794-bf7b-48d3-80b7-ebad3b90b6c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96732f1eeaf64fc8a23e8478c0c2f4c6', 'request_id': 'req-cc3f7794-bf7b-48d3-80b7-ebad3b90b6c3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:15.161143 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b2d444e4-1ebd-44fb-b421-68d56ad9aa0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89e8253124fe4830ac08d7c24a05efbf', 'request_id': 'req-b2d444e4-1ebd-44fb-b421-68d56ad9aa0f'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:15.244563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-cc3f7794-bf7b-48d3-80b7-ebad3b90b6c3 admin admin] The token cache is being invalidate because project 96732f1eeaf64fc8a23e8478c0c2f4c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:15.244853 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-cc3f7794-bf7b-48d3-80b7-ebad3b90b6c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:15.245456 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-cc3f7794-bf7b-48d3-80b7-ebad3b90b6c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:15.247038 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1223/2412] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:14 2022] DELETE /identity/v3/projects/96732f1eeaf64fc8a23e8478c0c2f4c6 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:15.247912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f7794-bf7b-48d3-80b7-ebad3b90b6c3 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:15.249680 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-cc3f7794-bf7b-48d3-80b7-ebad3b90b6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:15.308592 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f7794-bf7b-48d3-80b7-ebad3b90b6c3 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:15.319715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b2d444e4-1ebd-44fb-b421-68d56ad9aa0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89e8253124fe4830ac08d7c24a05efbf', 'request_id': 'req-b2d444e4-1ebd-44fb-b421-68d56ad9aa0f'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:15.326113 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b2d444e4-1ebd-44fb-b421-68d56ad9aa0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89e8253124fe4830ac08d7c24a05efbf', 'request_id': 'req-b2d444e4-1ebd-44fb-b421-68d56ad9aa0f'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:15.339085 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b2d444e4-1ebd-44fb-b421-68d56ad9aa0f admin admin] The token cache is being invalidate because project 89e8253124fe4830ac08d7c24a05efbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:15.339384 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b2d444e4-1ebd-44fb-b421-68d56ad9aa0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:15.340210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b2d444e4-1ebd-44fb-b421-68d56ad9aa0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:15.341822 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1190/2413] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:14 2022] DELETE /identity/v3/projects/89e8253124fe4830ac08d7c24a05efbf => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:19:15.343196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3003ae-23ae-489c-80eb-f2d9053867cf None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:15.343480 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3003ae-23ae-489c-80eb-f2d9053867cf None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:15.343800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3003ae-23ae-489c-80eb-f2d9053867cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:15.362638 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877bcf4-c09f-4902-8faf-cd5f6c48d4f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:15.363184 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-3e3003ae-23ae-489c-80eb-f2d9053867cf None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:15.363371 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877bcf4-c09f-4902-8faf-cd5f6c48d4f0 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:15.363708 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877bcf4-c09f-4902-8faf-cd5f6c48d4f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:15.364063 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877bcf4-c09f-4902-8faf-cd5f6c48d4f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:15.364983 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3e3003ae-23ae-489c-80eb-f2d9053867cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:15.366639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a877bcf4-c09f-4902-8faf-cd5f6c48d4f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:15.435378 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1191/2414] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:15.437951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679d615-60a7-49cb-ada9-b5863461b8e4 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:15.438321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679d615-60a7-49cb-ada9-b5863461b8e4 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:15.438670 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679d615-60a7-49cb-ada9-b5863461b8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:15.443716 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a877bcf4-c09f-4902-8faf-cd5f6c48d4f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:15.448559 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a877bcf4-c09f-4902-8faf-cd5f6c48d4f0 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:15.458046 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1224/2415] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:19:15.459448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c132355-caac-412e-b80a-d61e99a07085 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:15.459789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c132355-caac-412e-b80a-d61e99a07085 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:15.460032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c132355-caac-412e-b80a-d61e99a07085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:15.465838 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-e679d615-60a7-49cb-ada9-b5863461b8e4 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:15.467697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-e679d615-60a7-49cb-ada9-b5863461b8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:15.469387 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:15.480681 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1192/2416] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:15.483945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-7c132355-caac-412e-b80a-d61e99a07085 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:15.485299 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-7c132355-caac-412e-b80a-d61e99a07085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:15.488286 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751c3ac-db1f-47b1-aba7-948e2dfeb28e None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:15.488711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751c3ac-db1f-47b1-aba7-948e2dfeb28e None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:15.489715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751c3ac-db1f-47b1-aba7-948e2dfeb28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:15.497724 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1225/2417] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:15.498800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c132355-caac-412e-b80a-d61e99a07085 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:15.512986 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c132355-caac-412e-b80a-d61e99a07085 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:15.516302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-5751c3ac-db1f-47b1-aba7-948e2dfeb28e None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:15.517527 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-5751c3ac-db1f-47b1-aba7-948e2dfeb28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:15.521069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6222a1-dd3f-43f0-bc48-4c9667a642c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:15.521819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6222a1-dd3f-43f0-bc48-4c9667a642c6 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:15.522126 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6222a1-dd3f-43f0-bc48-4c9667a642c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:15.522409 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6222a1-dd3f-43f0-bc48-4c9667a642c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:15.523213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6222a1-dd3f-43f0-bc48-4c9667a642c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:19:15.523636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6222a1-dd3f-43f0-bc48-4c9667a642c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:15.525470 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6222a1-dd3f-43f0-bc48-4c9667a642c6 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:15.527946 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1193/2418] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:19:15.531749 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751c3ac-db1f-47b1-aba7-948e2dfeb28e None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:15.542793 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1226/2419] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:19:15.545271 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751c3ac-db1f-47b1-aba7-948e2dfeb28e None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:15.547137 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6222a1-dd3f-43f0-bc48-4c9667a642c6 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:15.550570 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443bb4c-9595-4d53-ab97-e8925d6170ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:15.551276 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443bb4c-9595-4d53-ab97-e8925d6170ae admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:15.552215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443bb4c-9595-4d53-ab97-e8925d6170ae admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:15.552726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443bb4c-9595-4d53-ab97-e8925d6170ae admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:15.553552 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443bb4c-9595-4d53-ab97-e8925d6170ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:19:15.553866 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443bb4c-9595-4d53-ab97-e8925d6170ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:15.555528 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443bb4c-9595-4d53-ab97-e8925d6170ae admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:15.558954 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6222a1-dd3f-43f0-bc48-4c9667a642c6 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:15.564576 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23062a77-8ffe-46d9-bb2c-b36aa39f06d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:15.565528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23062a77-8ffe-46d9-bb2c-b36aa39f06d1 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:15.565528 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23062a77-8ffe-46d9-bb2c-b36aa39f06d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:15.569389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23062a77-8ffe-46d9-bb2c-b36aa39f06d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:15.569389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23062a77-8ffe-46d9-bb2c-b36aa39f06d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:15.569389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23062a77-8ffe-46d9-bb2c-b36aa39f06d1 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:15.571612 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1194/2420] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:19:15.577741 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443bb4c-9595-4d53-ab97-e8925d6170ae admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:15.591441 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443bb4c-9595-4d53-ab97-e8925d6170ae admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:15.597699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad79a6f-34f7-4b97-bdcd-5e0bc977e036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:15.597699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad79a6f-34f7-4b97-bdcd-5e0bc977e036 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:15.597699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad79a6f-34f7-4b97-bdcd-5e0bc977e036 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:15.598062 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad79a6f-34f7-4b97-bdcd-5e0bc977e036 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:15.599327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad79a6f-34f7-4b97-bdcd-5e0bc977e036 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:15.601689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad79a6f-34f7-4b97-bdcd-5e0bc977e036 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:15.616896 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:15.663356 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1227/2421] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:19:15 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:19:15.667660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23062a77-8ffe-46d9-bb2c-b36aa39f06d1 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:15.736566 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23062a77-8ffe-46d9-bb2c-b36aa39f06d1 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:15.742034 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae499226-0f13-4dd5-8230-128457a794bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:15.742941 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae499226-0f13-4dd5-8230-128457a794bf admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:15.743322 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae499226-0f13-4dd5-8230-128457a794bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:15.743700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae499226-0f13-4dd5-8230-128457a794bf admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:15.743868 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1195/2422] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:19:15 2022] POST /identity/v3/projects => generated 387 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:19:15.744621 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae499226-0f13-4dd5-8230-128457a794bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:15.747095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae499226-0f13-4dd5-8230-128457a794bf admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:15.752821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad79a6f-34f7-4b97-bdcd-5e0bc977e036 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:15.818616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad79a6f-34f7-4b97-bdcd-5e0bc977e036 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:15.825561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2862d-4e4b-45fb-8739-8aead51c1453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:15.826258 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2862d-4e4b-45fb-8739-8aead51c1453 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:15.826437 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2862d-4e4b-45fb-8739-8aead51c1453 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:15.826700 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2862d-4e4b-45fb-8739-8aead51c1453 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:15.827456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2862d-4e4b-45fb-8739-8aead51c1453 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:15.829359 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2862d-4e4b-45fb-8739-8aead51c1453 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:15.855354 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1228/2423] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:19:15 2022] POST /identity/v3/users => generated 290 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:19:15.856870 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae499226-0f13-4dd5-8230-128457a794bf admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:15.867920 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae499226-0f13-4dd5-8230-128457a794bf admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:15.873168 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51c28f-f189-40be-9506-86501d2558bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:15.874725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51c28f-f189-40be-9506-86501d2558bf admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:15.875001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51c28f-f189-40be-9506-86501d2558bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:15.875272 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51c28f-f189-40be-9506-86501d2558bf admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:15.875914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51c28f-f189-40be-9506-86501d2558bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:15.877550 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51c28f-f189-40be-9506-86501d2558bf admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:15.889081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1229/2424] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:15.899087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51c28f-f189-40be-9506-86501d2558bf admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:15.909792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51c28f-f189-40be-9506-86501d2558bf admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:15.914450 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1196/2425] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:19:15 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:19:15.915050 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880c713-5ad9-4dea-96e2-cf98bab274ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:15.915663 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880c713-5ad9-4dea-96e2-cf98bab274ed admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:15.915988 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880c713-5ad9-4dea-96e2-cf98bab274ed admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:15.916303 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880c713-5ad9-4dea-96e2-cf98bab274ed admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:15.918330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f880c713-5ad9-4dea-96e2-cf98bab274ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:15.918814 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f880c713-5ad9-4dea-96e2-cf98bab274ed admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:15.918950 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2862d-4e4b-45fb-8739-8aead51c1453 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:15.921520 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:15.932219 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2862d-4e4b-45fb-8739-8aead51c1453 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:15.939535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfccf06f-7147-4f36-baa0-0374cae6cc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:15.940487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1230/2426] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:19:15.940949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccf06f-7147-4f36-baa0-0374cae6cc71 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:15.941295 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccf06f-7147-4f36-baa0-0374cae6cc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:15.941738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccf06f-7147-4f36-baa0-0374cae6cc71 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:15.943059 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfccf06f-7147-4f36-baa0-0374cae6cc71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:15.945674 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880c713-5ad9-4dea-96e2-cf98bab274ed admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:15.945892 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfccf06f-7147-4f36-baa0-0374cae6cc71 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:15.961480 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1197/2427] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:15.961729 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880c713-5ad9-4dea-96e2-cf98bab274ed admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:15.962853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfccf06f-7147-4f36-baa0-0374cae6cc71 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:15.965826 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d4ded-220d-456b-8b31-e6b6ffeac49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:15.966635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d4ded-220d-456b-8b31-e6b6ffeac49a admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:15.966906 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d4ded-220d-456b-8b31-e6b6ffeac49a admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:15.967240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d4ded-220d-456b-8b31-e6b6ffeac49a admin admin] PATH_INFO: `/v3/projects/65d0e5c61cc54ec48d96c75060928e88/users/4bc12706fb5d4222a0da74ee2026a671/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:15.973372 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfccf06f-7147-4f36-baa0-0374cae6cc71 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:15.979890 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee6531-9545-4fc2-9720-5e3ec3c0f40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:15.980539 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee6531-9545-4fc2-9720-5e3ec3c0f40d admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:15.980819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee6531-9545-4fc2-9720-5e3ec3c0f40d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:15.981096 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee6531-9545-4fc2-9720-5e3ec3c0f40d admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:15.981830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee6531-9545-4fc2-9720-5e3ec3c0f40d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:15.983421 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee6531-9545-4fc2-9720-5e3ec3c0f40d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:15.990292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d4ded-220d-456b-8b31-e6b6ffeac49a admin admin] RBAC: Authorizing `identity:create_grant(project_id=65d0e5c61cc54ec48d96c75060928e88, user_id=4bc12706fb5d4222a0da74ee2026a671, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:15.994108 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d4ded-220d-456b-8b31-e6b6ffeac49a admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:15.995133 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1198/2428] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:15.998999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee6531-9545-4fc2-9720-5e3ec3c0f40d admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:16.010161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee6531-9545-4fc2-9720-5e3ec3c0f40d admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:16.015143 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12b1e2-6564-4541-beb7-c36f8c699576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:16.015829 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12b1e2-6564-4541-beb7-c36f8c699576 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:16.016167 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12b1e2-6564-4541-beb7-c36f8c699576 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:16.016442 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12b1e2-6564-4541-beb7-c36f8c699576 admin admin] PATH_INFO: `/v3/projects/f4b2e8a457c14f628f6c20a748ccb727/users/ee504c6c96da4b8fa26202928a3b62f5/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:16.035637 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12b1e2-6564-4541-beb7-c36f8c699576 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b2e8a457c14f628f6c20a748ccb727, user_id=ee504c6c96da4b8fa26202928a3b62f5, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:16.038753 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12b1e2-6564-4541-beb7-c36f8c699576 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:16.045329 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1231/2429] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:15 2022] PUT /identity/v3/projects/65d0e5c61cc54ec48d96c75060928e88/users/4bc12706fb5d4222a0da74ee2026a671/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:19:16.061527 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fe306-b71d-40c7-8e37-6a381371127e None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:16.062028 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fe306-b71d-40c7-8e37-6a381371127e None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:16.062480 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fe306-b71d-40c7-8e37-6a381371127e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:16.096304 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-8f4fe306-b71d-40c7-8e37-6a381371127e None None] MFA Rules not processed for user `4bc12706fb5d4222a0da74ee2026a671`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:16.097567 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8f4fe306-b71d-40c7-8e37-6a381371127e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:16.111278 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1199/2430] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:15 2022] PUT /identity/v3/projects/f4b2e8a457c14f628f6c20a748ccb727/users/ee504c6c96da4b8fa26202928a3b62f5/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:19:16.123658 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7ee30-dc73-4523-86be-e28457ca2bbd None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:16.123996 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7ee30-dc73-4523-86be-e28457ca2bbd None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:16.124216 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7ee30-dc73-4523-86be-e28457ca2bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:16.163469 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-c0a7ee30-dc73-4523-86be-e28457ca2bbd None None] MFA Rules not processed for user `ee504c6c96da4b8fa26202928a3b62f5`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:16.165028 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-c0a7ee30-dc73-4523-86be-e28457ca2bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:16.190359 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1232/2431] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:19:16.195366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fe306-b71d-40c7-8e37-6a381371127e None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:16.239065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1200/2432] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:16.244401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a7ee30-dc73-4523-86be-e28457ca2bbd None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:16.275218 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fe306-b71d-40c7-8e37-6a381371127e None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:16.279401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:16.361369 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a7ee30-dc73-4523-86be-e28457ca2bbd None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:16.370152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448c64d-b434-4f73-8958-76b0b5f6927a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:16.371004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448c64d-b434-4f73-8958-76b0b5f6927a admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:16.371137 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675c791-dc07-4bd8-8533-afefbef569c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:16.371551 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448c64d-b434-4f73-8958-76b0b5f6927a admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:16.371922 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675c791-dc07-4bd8-8533-afefbef569c3 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:16.372092 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448c64d-b434-4f73-8958-76b0b5f6927a admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:16.372267 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675c791-dc07-4bd8-8533-afefbef569c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:16.372565 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675c791-dc07-4bd8-8533-afefbef569c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:16.373083 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448c64d-b434-4f73-8958-76b0b5f6927a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:16.373227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675c791-dc07-4bd8-8533-afefbef569c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:16.375398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448c64d-b434-4f73-8958-76b0b5f6927a admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:16.376335 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675c791-dc07-4bd8-8533-afefbef569c3 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:16.397246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:16.507580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1201/2433] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:19:16 2022] POST /identity/v3/projects => generated 387 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:19:16.507824 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1233/2434] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:19:16 2022] POST /identity/v3/projects => generated 379 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:19:16.509602 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675c791-dc07-4bd8-8533-afefbef569c3 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:16.510215 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448c64d-b434-4f73-8958-76b0b5f6927a admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:16.589614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675c791-dc07-4bd8-8533-afefbef569c3 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:16.607729 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448c64d-b434-4f73-8958-76b0b5f6927a admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:16.614945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab91275-8074-407e-b0a4-0d1eb5bc35f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:16.615696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab91275-8074-407e-b0a4-0d1eb5bc35f3 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:16.616065 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab91275-8074-407e-b0a4-0d1eb5bc35f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:16.616526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab91275-8074-407e-b0a4-0d1eb5bc35f3 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:16.621069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab91275-8074-407e-b0a4-0d1eb5bc35f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:16.621069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab91275-8074-407e-b0a4-0d1eb5bc35f3 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:16.662324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2319d-3e99-43d2-b4e3-b69225dfcf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:16.662964 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2319d-3e99-43d2-b4e3-b69225dfcf63 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:16.663486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2319d-3e99-43d2-b4e3-b69225dfcf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:16.663769 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2319d-3e99-43d2-b4e3-b69225dfcf63 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:16.664514 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff2319d-3e99-43d2-b4e3-b69225dfcf63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:16.666942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff2319d-3e99-43d2-b4e3-b69225dfcf63 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:16.715914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1234/2435] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:19:16 2022] POST /identity/v3/users => generated 296 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:19:16.720401 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab91275-8074-407e-b0a4-0d1eb5bc35f3 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:16.732746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab91275-8074-407e-b0a4-0d1eb5bc35f3 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:16.744316 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad946c-cf13-48b6-bfbf-48a2e6ba9845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:16.745464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad946c-cf13-48b6-bfbf-48a2e6ba9845 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:16.745910 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad946c-cf13-48b6-bfbf-48a2e6ba9845 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:16.746364 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad946c-cf13-48b6-bfbf-48a2e6ba9845 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:16.747185 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad946c-cf13-48b6-bfbf-48a2e6ba9845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:16.749350 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad946c-cf13-48b6-bfbf-48a2e6ba9845 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:16.767116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1235/2436] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:16.771421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad946c-cf13-48b6-bfbf-48a2e6ba9845 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:16.784384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad946c-cf13-48b6-bfbf-48a2e6ba9845 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:16.792707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0984bfe-959b-46d1-8c97-2c714ace185e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:16.794103 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0984bfe-959b-46d1-8c97-2c714ace185e admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:16.794665 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0984bfe-959b-46d1-8c97-2c714ace185e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:16.795304 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0984bfe-959b-46d1-8c97-2c714ace185e admin admin] PATH_INFO: `/v3/projects/08c24e75698a4c30820b12cb1315d791/users/8b1a9ad4d9114098b14698039c2d445e/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:16.800187 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1202/2437] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:19:16 2022] POST /identity/v3/users => generated 300 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:19:16.804390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2319d-3e99-43d2-b4e3-b69225dfcf63 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:16.826982 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2319d-3e99-43d2-b4e3-b69225dfcf63 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:16.834380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0984bfe-959b-46d1-8c97-2c714ace185e admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c24e75698a4c30820b12cb1315d791, user_id=8b1a9ad4d9114098b14698039c2d445e, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:16.837273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0984bfe-959b-46d1-8c97-2c714ace185e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:16.838141 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a54f2-efaa-41d9-8a31-e8b4fb79617d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:16.842771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a54f2-efaa-41d9-8a31-e8b4fb79617d admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:16.842771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a54f2-efaa-41d9-8a31-e8b4fb79617d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:16.842771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a54f2-efaa-41d9-8a31-e8b4fb79617d admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:16.842771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a54f2-efaa-41d9-8a31-e8b4fb79617d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:16.843485 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a54f2-efaa-41d9-8a31-e8b4fb79617d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:16.856824 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1203/2438] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:19:16.860922 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a54f2-efaa-41d9-8a31-e8b4fb79617d admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:16.873274 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a54f2-efaa-41d9-8a31-e8b4fb79617d admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:16.879012 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126daf2-b47d-4ad0-991f-4d7b7002edc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:16.880481 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126daf2-b47d-4ad0-991f-4d7b7002edc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:16.880979 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126daf2-b47d-4ad0-991f-4d7b7002edc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:16.881116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126daf2-b47d-4ad0-991f-4d7b7002edc9 admin admin] PATH_INFO: `/v3/projects/65cf0f36eb214b9b90daf93dd8d587ac/users/a409bb5f644d4dc0beb3080dfa3e72fd/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:16.902349 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126daf2-b47d-4ad0-991f-4d7b7002edc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65cf0f36eb214b9b90daf93dd8d587ac, user_id=a409bb5f644d4dc0beb3080dfa3e72fd, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:16.902585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126daf2-b47d-4ad0-991f-4d7b7002edc9 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:16.922091 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1236/2439] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:16 2022] PUT /identity/v3/projects/08c24e75698a4c30820b12cb1315d791/users/8b1a9ad4d9114098b14698039c2d445e/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:16.927380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0984bfe-959b-46d1-8c97-2c714ace185e admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:16.986255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1204/2440] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:16 2022] PUT /identity/v3/projects/65cf0f36eb214b9b90daf93dd8d587ac/users/a409bb5f644d4dc0beb3080dfa3e72fd/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:19:16.991595 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126daf2-b47d-4ad0-991f-4d7b7002edc9 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:17.011958 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:17.034461 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0984bfe-959b-46d1-8c97-2c714ace185e admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:17.096151 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126daf2-b47d-4ad0-991f-4d7b7002edc9 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:17.104406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10403e6-d2c9-4a47-bb4b-9d88d46a4364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:17.105200 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10403e6-d2c9-4a47-bb4b-9d88d46a4364 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:17.105625 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10403e6-d2c9-4a47-bb4b-9d88d46a4364 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:17.106010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10403e6-d2c9-4a47-bb4b-9d88d46a4364 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:17.106781 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10403e6-d2c9-4a47-bb4b-9d88d46a4364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:17.111406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10403e6-d2c9-4a47-bb4b-9d88d46a4364 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:17.120914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:17.128703 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1205/2441] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:17.130331 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10403e6-d2c9-4a47-bb4b-9d88d46a4364 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:17.134996 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981d668-937c-49f7-b903-65a0aba5bb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:17.136081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981d668-937c-49f7-b903-65a0aba5bb61 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:17.136639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981d668-937c-49f7-b903-65a0aba5bb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:17.138399 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981d668-937c-49f7-b903-65a0aba5bb61 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:17.138399 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981d668-937c-49f7-b903-65a0aba5bb61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:17.140048 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981d668-937c-49f7-b903-65a0aba5bb61 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:17.153977 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1237/2442] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:17.155255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10403e6-d2c9-4a47-bb4b-9d88d46a4364 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:17.158871 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981d668-937c-49f7-b903-65a0aba5bb61 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:17.162361 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bab929-7781-4b1b-b157-d4be338fa4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:17.163621 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bab929-7781-4b1b-b157-d4be338fa4e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:17.163899 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bab929-7781-4b1b-b157-d4be338fa4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:17.164351 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bab929-7781-4b1b-b157-d4be338fa4e9 admin admin] PATH_INFO: `/v3/projects/65cf0f36eb214b9b90daf93dd8d587ac/users/a409bb5f644d4dc0beb3080dfa3e72fd/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:17.167242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bab929-7781-4b1b-b157-d4be338fa4e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65cf0f36eb214b9b90daf93dd8d587ac, user_id=a409bb5f644d4dc0beb3080dfa3e72fd, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:17.171653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bab929-7781-4b1b-b157-d4be338fa4e9 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:17.173540 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981d668-937c-49f7-b903-65a0aba5bb61 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:17.180201 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbdf4a9-3166-4dd7-a59f-c9861cd07ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:17.181177 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdf4a9-3166-4dd7-a59f-c9861cd07ef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:17.181524 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdf4a9-3166-4dd7-a59f-c9861cd07ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:17.182051 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdf4a9-3166-4dd7-a59f-c9861cd07ef3 admin admin] PATH_INFO: `/v3/projects/08c24e75698a4c30820b12cb1315d791/users/8b1a9ad4d9114098b14698039c2d445e/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:17.184399 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbdf4a9-3166-4dd7-a59f-c9861cd07ef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c24e75698a4c30820b12cb1315d791, user_id=8b1a9ad4d9114098b14698039c2d445e, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:17.187295 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbdf4a9-3166-4dd7-a59f-c9861cd07ef3 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:17.198202 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1206/2443] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:17 2022] PUT /identity/v3/projects/65cf0f36eb214b9b90daf93dd8d587ac/users/a409bb5f644d4dc0beb3080dfa3e72fd/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:19:17.199705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c039c-f119-4eb2-8934-20ee2c4d516d None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:17.199995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c039c-f119-4eb2-8934-20ee2c4d516d None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:17.200299 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c039c-f119-4eb2-8934-20ee2c4d516d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:17.213086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1238/2444] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:17 2022] PUT /identity/v3/projects/08c24e75698a4c30820b12cb1315d791/users/8b1a9ad4d9114098b14698039c2d445e/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:17.214205 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ade1c6-491f-4a2d-9951-365d57009bb3 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:17.215535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ade1c6-491f-4a2d-9951-365d57009bb3 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:17.215535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ade1c6-491f-4a2d-9951-365d57009bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:17.269938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-262c039c-f119-4eb2-8934-20ee2c4d516d None None] MFA Rules not processed for user `a409bb5f644d4dc0beb3080dfa3e72fd`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:17.278287 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-262c039c-f119-4eb2-8934-20ee2c4d516d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:17.280732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-c6ade1c6-491f-4a2d-9951-365d57009bb3 None None] MFA Rules not processed for user `8b1a9ad4d9114098b14698039c2d445e`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:17.282609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c6ade1c6-491f-4a2d-9951-365d57009bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:17.398655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1239/2445] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:17 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:17.398655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ade1c6-491f-4a2d-9951-365d57009bb3 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:17.400494 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c6ade1c6-491f-4a2d-9951-365d57009bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:17.402267 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1207/2446] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:17 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:17.416760 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c039c-f119-4eb2-8934-20ee2c4d516d None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:17.418208 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-262c039c-f119-4eb2-8934-20ee2c4d516d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:17.524612 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ade1c6-491f-4a2d-9951-365d57009bb3 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:17.536887 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38265e1-a553-490c-bc16-f972f0389588 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:17.537902 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38265e1-a553-490c-bc16-f972f0389588 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:17.538648 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38265e1-a553-490c-bc16-f972f0389588 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:17.539140 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38265e1-a553-490c-bc16-f972f0389588 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:17.541474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c38265e1-a553-490c-bc16-f972f0389588 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:17.564094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38265e1-a553-490c-bc16-f972f0389588 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:17.566004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38265e1-a553-490c-bc16-f972f0389588 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:17.605661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1240/2447] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:19:17.607861 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c039c-f119-4eb2-8934-20ee2c4d516d None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:17.608627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38265e1-a553-490c-bc16-f972f0389588 service nova] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:17.611256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c38265e1-a553-490c-bc16-f972f0389588 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:17.624040 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8630e-97f3-4390-931e-079957e858fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:17.624809 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8630e-97f3-4390-931e-079957e858fe service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:17.625093 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8630e-97f3-4390-931e-079957e858fe service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:17.625452 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8630e-97f3-4390-931e-079957e858fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:17.640163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38265e1-a553-490c-bc16-f972f0389588 service nova] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:17.643712 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8630e-97f3-4390-931e-079957e858fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:17.645823 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8630e-97f3-4390-931e-079957e858fe service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:17.649622 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d0826-4872-4a4c-a01a-5203caec9280 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:17.651961 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d0826-4872-4a4c-a01a-5203caec9280 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:17.651961 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d0826-4872-4a4c-a01a-5203caec9280 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:17.652204 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d0826-4872-4a4c-a01a-5203caec9280 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:17.655641 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a93d0826-4872-4a4c-a01a-5203caec9280 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:17.658939 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1208/2448] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:17 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:19:17.699885 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d0826-4872-4a4c-a01a-5203caec9280 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:17.701978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d0826-4872-4a4c-a01a-5203caec9280 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:17.728288 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1241/2449] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:19:17.750208 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8630e-97f3-4390-931e-079957e858fe service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:17.763915 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8630e-97f3-4390-931e-079957e858fe service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:17.777779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd2e09-d148-440c-8ce1-cd8ea5ad5d45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:17.781695 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd2e09-d148-440c-8ce1-cd8ea5ad5d45 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:17.781695 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd2e09-d148-440c-8ce1-cd8ea5ad5d45 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:17.781695 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd2e09-d148-440c-8ce1-cd8ea5ad5d45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:17.798731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd2e09-d148-440c-8ce1-cd8ea5ad5d45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:17.802755 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd2e09-d148-440c-8ce1-cd8ea5ad5d45 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:17.851054 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1209/2450] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:17 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:19:19.291793 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d0826-4872-4a4c-a01a-5203caec9280 service nova] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:19.317886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d0826-4872-4a4c-a01a-5203caec9280 service nova] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:19.326450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a835c8-195b-4670-86f4-114ef373dfea tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1a9ad4d9114098b14698039c2d445e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08c24e75698a4c30820b12cb1315d791', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:19.331045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a835c8-195b-4670-86f4-114ef373dfea tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:19.331045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a835c8-195b-4670-86f4-114ef373dfea tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:19.331045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a835c8-195b-4670-86f4-114ef373dfea tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:19.331045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a835c8-195b-4670-86f4-114ef373dfea tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:19:19.331045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a835c8-195b-4670-86f4-114ef373dfea tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:19.331492 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a835c8-195b-4670-86f4-114ef373dfea tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:19.359563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1242/2451] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:19:19.360712 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd2e09-d148-440c-8ce1-cd8ea5ad5d45 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:19.372836 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd2e09-d148-440c-8ce1-cd8ea5ad5d45 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:19.377715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55234352-4046-40dd-994b-f734a2b0a93a tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1a9ad4d9114098b14698039c2d445e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08c24e75698a4c30820b12cb1315d791', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:19.378396 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55234352-4046-40dd-994b-f734a2b0a93a tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:19.383682 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55234352-4046-40dd-994b-f734a2b0a93a tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:19.383682 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55234352-4046-40dd-994b-f734a2b0a93a tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:19.383682 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55234352-4046-40dd-994b-f734a2b0a93a tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:19.383682 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55234352-4046-40dd-994b-f734a2b0a93a tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:19.388788 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:19.449313 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a835c8-195b-4670-86f4-114ef373dfea tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:19.475401 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:19.499622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1210/2452] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:19:19 2022] POST /identity/v3/projects => generated 354 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:19:19.500560 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55234352-4046-40dd-994b-f734a2b0a93a tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:19.522370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a835c8-195b-4670-86f4-114ef373dfea tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:19.567927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55234352-4046-40dd-994b-f734a2b0a93a tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:19.572840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f535f4-2a5f-49bf-a493-56d11fb85730 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1a9ad4d9114098b14698039c2d445e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08c24e75698a4c30820b12cb1315d791', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:19.573518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f535f4-2a5f-49bf-a493-56d11fb85730 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:19.573800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f535f4-2a5f-49bf-a493-56d11fb85730 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:19.574075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f535f4-2a5f-49bf-a493-56d11fb85730 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:19.574720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f535f4-2a5f-49bf-a493-56d11fb85730 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:19:19.575022 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f535f4-2a5f-49bf-a493-56d11fb85730 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:19.576639 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f535f4-2a5f-49bf-a493-56d11fb85730 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:19.588418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cfc2a-0754-4397-aeef-135f21e8245a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:19.588418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cfc2a-0754-4397-aeef-135f21e8245a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:19.588418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cfc2a-0754-4397-aeef-135f21e8245a admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:19.588418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cfc2a-0754-4397-aeef-135f21e8245a admin admin] PATH_INFO: `/v3/users/ee504c6c96da4b8fa26202928a3b62f5` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:19.589475 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1211/2453] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:19:19.601669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2cfc2a-0754-4397-aeef-135f21e8245a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee504c6c96da4b8fa26202928a3b62f5)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:19.610130 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2cfc2a-0754-4397-aeef-135f21e8245a admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:19.630430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f535f4-2a5f-49bf-a493-56d11fb85730 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:19.645326 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f535f4-2a5f-49bf-a493-56d11fb85730 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:19.650936 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581cc7cf-2806-4a50-920c-837dad6e2725 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b1a9ad4d9114098b14698039c2d445e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08c24e75698a4c30820b12cb1315d791', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:19.651580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cc7cf-2806-4a50-920c-837dad6e2725 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:19.651860 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cc7cf-2806-4a50-920c-837dad6e2725 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:19.652163 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cc7cf-2806-4a50-920c-837dad6e2725 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] PATH_INFO: `/v3/projects/e581a229e1e3457c8522b62b1bac4939` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:19.653856 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581cc7cf-2806-4a50-920c-837dad6e2725 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e581a229e1e3457c8522b62b1bac4939)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:19.655431 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581cc7cf-2806-4a50-920c-837dad6e2725 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:19.847832 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-581cc7cf-2806-4a50-920c-837dad6e2725 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e581a229e1e3457c8522b62b1bac4939', 'request_id': 'req-581cc7cf-2806-4a50-920c-837dad6e2725'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:19.850544 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-dd2cfc2a-0754-4397-aeef-135f21e8245a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee504c6c96da4b8fa26202928a3b62f5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:19.851445 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-dd2cfc2a-0754-4397-aeef-135f21e8245a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee504c6c96da4b8fa26202928a3b62f5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:19.858622 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-dd2cfc2a-0754-4397-aeef-135f21e8245a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee504c6c96da4b8fa26202928a3b62f5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:19.924111 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-581cc7cf-2806-4a50-920c-837dad6e2725 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e581a229e1e3457c8522b62b1bac4939', 'request_id': 'req-581cc7cf-2806-4a50-920c-837dad6e2725'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:19.931495 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-581cc7cf-2806-4a50-920c-837dad6e2725 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e581a229e1e3457c8522b62b1bac4939', 'request_id': 'req-581cc7cf-2806-4a50-920c-837dad6e2725'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:19.943790 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-581cc7cf-2806-4a50-920c-837dad6e2725 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] The token cache is being invalidate because project e581a229e1e3457c8522b62b1bac4939 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:19.944106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-581cc7cf-2806-4a50-920c-837dad6e2725 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:19.944868 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-581cc7cf-2806-4a50-920c-837dad6e2725 tempest-VolumeQuotaClassesTest-744241661 tempest-VolumeQuotaClassesTest-744241661-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:19.946507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1212/2454] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:19 2022] DELETE /identity/v3/projects/e581a229e1e3457c8522b62b1bac4939 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:20.003860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-dd2cfc2a-0754-4397-aeef-135f21e8245a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee504c6c96da4b8fa26202928a3b62f5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:20.004626 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-dd2cfc2a-0754-4397-aeef-135f21e8245a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee504c6c96da4b8fa26202928a3b62f5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:20.024493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1243/2455] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:19:19 2022] DELETE /identity/v3/users/ee504c6c96da4b8fa26202928a3b62f5 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:19:20.029171 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e9508-b9a6-4daf-9779-d76da4d239da None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:20.029453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e9508-b9a6-4daf-9779-d76da4d239da None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:20.029721 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e9508-b9a6-4daf-9779-d76da4d239da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:20.050434 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-ff4e9508-b9a6-4daf-9779-d76da4d239da None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:20.051653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ff4e9508-b9a6-4daf-9779-d76da4d239da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:20.113250 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1213/2456] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:19:20.123445 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cfc2a-0754-4397-aeef-135f21e8245a admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:20.124971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-dd2cfc2a-0754-4397-aeef-135f21e8245a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:20.180660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:20.184417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cfc2a-0754-4397-aeef-135f21e8245a admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:20.187980 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694d6ac-d6f6-4be5-bf2c-a06dba8e1879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:20.188617 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694d6ac-d6f6-4be5-bf2c-a06dba8e1879 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:20.188883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694d6ac-d6f6-4be5-bf2c-a06dba8e1879 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:20.189152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694d6ac-d6f6-4be5-bf2c-a06dba8e1879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:20.198837 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694d6ac-d6f6-4be5-bf2c-a06dba8e1879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:20.200339 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694d6ac-d6f6-4be5-bf2c-a06dba8e1879 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:20.221894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1244/2457] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:19:20.597098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e9508-b9a6-4daf-9779-d76da4d239da None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:20.598532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ff4e9508-b9a6-4daf-9779-d76da4d239da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:20.612782 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:20.612782 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e9508-b9a6-4daf-9779-d76da4d239da None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:20.617087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694d6ac-d6f6-4be5-bf2c-a06dba8e1879 service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:20.618287 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bedfa-bcc3-442c-8f21-8e7631478f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:20.618892 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bedfa-bcc3-442c-8f21-8e7631478f59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:20.619155 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bedfa-bcc3-442c-8f21-8e7631478f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:20.620194 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bedfa-bcc3-442c-8f21-8e7631478f59 admin admin] PATH_INFO: `/v3/users/4bc12706fb5d4222a0da74ee2026a671` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:20.621039 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0bedfa-bcc3-442c-8f21-8e7631478f59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bc12706fb5d4222a0da74ee2026a671)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:20.621569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a694d6ac-d6f6-4be5-bf2c-a06dba8e1879 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:20.623397 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0bedfa-bcc3-442c-8f21-8e7631478f59 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:20.641219 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694d6ac-d6f6-4be5-bf2c-a06dba8e1879 service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:20.648894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39a88c-666d-422c-b8ae-2d55085c2ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:20.649679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39a88c-666d-422c-b8ae-2d55085c2ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:20.650031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39a88c-666d-422c-b8ae-2d55085c2ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:20.650377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39a88c-666d-422c-b8ae-2d55085c2ae3 admin admin] PATH_INFO: `/v3/projects/f4b2e8a457c14f628f6c20a748ccb727` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:20.653213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39a88c-666d-422c-b8ae-2d55085c2ae3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4b2e8a457c14f628f6c20a748ccb727)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:20.658317 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39a88c-666d-422c-b8ae-2d55085c2ae3 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:20.876110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0c0bedfa-bcc3-442c-8f21-8e7631478f59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bc12706fb5d4222a0da74ee2026a671'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:20.878134 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-2f39a88c-666d-422c-b8ae-2d55085c2ae3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4b2e8a457c14f628f6c20a748ccb727', 'request_id': 'req-2f39a88c-666d-422c-b8ae-2d55085c2ae3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:20.884263 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0c0bedfa-bcc3-442c-8f21-8e7631478f59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bc12706fb5d4222a0da74ee2026a671'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:20.885022 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0c0bedfa-bcc3-442c-8f21-8e7631478f59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bc12706fb5d4222a0da74ee2026a671'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:20.885850 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-2f39a88c-666d-422c-b8ae-2d55085c2ae3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4b2e8a457c14f628f6c20a748ccb727', 'request_id': 'req-2f39a88c-666d-422c-b8ae-2d55085c2ae3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:20.892042 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-2f39a88c-666d-422c-b8ae-2d55085c2ae3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4b2e8a457c14f628f6c20a748ccb727', 'request_id': 'req-2f39a88c-666d-422c-b8ae-2d55085c2ae3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:20.898275 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0c0bedfa-bcc3-442c-8f21-8e7631478f59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bc12706fb5d4222a0da74ee2026a671'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:20.899037 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-0c0bedfa-bcc3-442c-8f21-8e7631478f59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bc12706fb5d4222a0da74ee2026a671'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:20.964343 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-2f39a88c-666d-422c-b8ae-2d55085c2ae3 admin admin] The token cache is being invalidate because project f4b2e8a457c14f628f6c20a748ccb727 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:20.964637 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-2f39a88c-666d-422c-b8ae-2d55085c2ae3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:20.965248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-2f39a88c-666d-422c-b8ae-2d55085c2ae3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:20.966745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1245/2458] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:20 2022] DELETE /identity/v3/projects/f4b2e8a457c14f628f6c20a748ccb727 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:20.970666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39a88c-666d-422c-b8ae-2d55085c2ae3 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:20.975249 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-2f39a88c-666d-422c-b8ae-2d55085c2ae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:21.035904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1214/2459] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:19:20 2022] DELETE /identity/v3/users/4bc12706fb5d4222a0da74ee2026a671 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:21.039801 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39a88c-666d-422c-b8ae-2d55085c2ae3 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:21.040752 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7cb56-d1a7-4042-8992-ca1a0b34215f None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:21.041335 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7cb56-d1a7-4042-8992-ca1a0b34215f None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:21.041608 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7cb56-d1a7-4042-8992-ca1a0b34215f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:21.061483 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-80d7cb56-d1a7-4042-8992-ca1a0b34215f None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:21.062963 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-80d7cb56-d1a7-4042-8992-ca1a0b34215f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:21.105068 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9a493-9421-4f93-998a-c023a37845de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:21.105782 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9a493-9421-4f93-998a-c023a37845de admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:21.106191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9a493-9421-4f93-998a-c023a37845de admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:21.106545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9a493-9421-4f93-998a-c023a37845de admin admin] PATH_INFO: `/v3/users/a409bb5f644d4dc0beb3080dfa3e72fd` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:21.108238 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9a493-9421-4f93-998a-c023a37845de admin admin] RBAC: Authorizing `identity:delete_user(user_id=a409bb5f644d4dc0beb3080dfa3e72fd)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:21.110533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9a493-9421-4f93-998a-c023a37845de admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:21.129627 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1215/2460] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:19:21.140382 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:21.143635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7cb56-d1a7-4042-8992-ca1a0b34215f None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:21.145135 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-80d7cb56-d1a7-4042-8992-ca1a0b34215f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:21.202330 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7cb56-d1a7-4042-8992-ca1a0b34215f None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:21.206262 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0ee2d-ebbc-4eda-9397-27542edcc9ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:21.206899 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0ee2d-ebbc-4eda-9397-27542edcc9ec service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:21.207151 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0ee2d-ebbc-4eda-9397-27542edcc9ec service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:21.207426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0ee2d-ebbc-4eda-9397-27542edcc9ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:21.226005 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0ee2d-ebbc-4eda-9397-27542edcc9ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:21.227411 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0ee2d-ebbc-4eda-9397-27542edcc9ec service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:21.242540 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1216/2461] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:19:21.336005 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-02a9a493-9421-4f93-998a-c023a37845de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a409bb5f644d4dc0beb3080dfa3e72fd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:21.336740 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-02a9a493-9421-4f93-998a-c023a37845de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a409bb5f644d4dc0beb3080dfa3e72fd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:21.345927 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-02a9a493-9421-4f93-998a-c023a37845de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a409bb5f644d4dc0beb3080dfa3e72fd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:21.436525 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-02a9a493-9421-4f93-998a-c023a37845de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a409bb5f644d4dc0beb3080dfa3e72fd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:21.437171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-02a9a493-9421-4f93-998a-c023a37845de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a409bb5f644d4dc0beb3080dfa3e72fd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:21.455559 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1246/2462] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:19:20 2022] DELETE /identity/v3/users/a409bb5f644d4dc0beb3080dfa3e72fd => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:19:21.916646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0ee2d-ebbc-4eda-9397-27542edcc9ec service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:21.930915 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0ee2d-ebbc-4eda-9397-27542edcc9ec service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:21.962621 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9a493-9421-4f93-998a-c023a37845de admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:21.962621 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-02a9a493-9421-4f93-998a-c023a37845de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:21.965805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:22.061597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9a493-9421-4f93-998a-c023a37845de admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:22.068732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b09214-010a-42d3-885d-23d8f6f236b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:22.069959 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b09214-010a-42d3-885d-23d8f6f236b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:22.070532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b09214-010a-42d3-885d-23d8f6f236b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:22.071109 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b09214-010a-42d3-885d-23d8f6f236b7 admin admin] PATH_INFO: `/v3/projects/65d0e5c61cc54ec48d96c75060928e88` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:22.073590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b09214-010a-42d3-885d-23d8f6f236b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65d0e5c61cc54ec48d96c75060928e88)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:22.075914 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b09214-010a-42d3-885d-23d8f6f236b7 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:22.087566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4fc89d-f7ed-42c0-9a9b-87969dbf4c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:22.088266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fc89d-f7ed-42c0-9a9b-87969dbf4c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:22.088552 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fc89d-f7ed-42c0-9a9b-87969dbf4c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:22.088816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fc89d-f7ed-42c0-9a9b-87969dbf4c02 admin admin] PATH_INFO: `/v3/projects/65cf0f36eb214b9b90daf93dd8d587ac` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:22.095186 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4fc89d-f7ed-42c0-9a9b-87969dbf4c02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65cf0f36eb214b9b90daf93dd8d587ac)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:22.104108 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4fc89d-f7ed-42c0-9a9b-87969dbf4c02 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:22.262192 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-f3b09214-010a-42d3-885d-23d8f6f236b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65d0e5c61cc54ec48d96c75060928e88', 'request_id': 'req-f3b09214-010a-42d3-885d-23d8f6f236b7'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:22.263134 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-dd4fc89d-f7ed-42c0-9a9b-87969dbf4c02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65cf0f36eb214b9b90daf93dd8d587ac', 'request_id': 'req-dd4fc89d-f7ed-42c0-9a9b-87969dbf4c02'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:22.269234 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-f3b09214-010a-42d3-885d-23d8f6f236b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65d0e5c61cc54ec48d96c75060928e88', 'request_id': 'req-f3b09214-010a-42d3-885d-23d8f6f236b7'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:22.276146 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-f3b09214-010a-42d3-885d-23d8f6f236b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65d0e5c61cc54ec48d96c75060928e88', 'request_id': 'req-f3b09214-010a-42d3-885d-23d8f6f236b7'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:22.352780 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-dd4fc89d-f7ed-42c0-9a9b-87969dbf4c02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65cf0f36eb214b9b90daf93dd8d587ac', 'request_id': 'req-dd4fc89d-f7ed-42c0-9a9b-87969dbf4c02'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:22.359691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-dd4fc89d-f7ed-42c0-9a9b-87969dbf4c02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65cf0f36eb214b9b90daf93dd8d587ac', 'request_id': 'req-dd4fc89d-f7ed-42c0-9a9b-87969dbf4c02'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:22.372931 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-dd4fc89d-f7ed-42c0-9a9b-87969dbf4c02 admin admin] The token cache is being invalidate because project 65cf0f36eb214b9b90daf93dd8d587ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:22.373057 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-dd4fc89d-f7ed-42c0-9a9b-87969dbf4c02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:22.373810 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-dd4fc89d-f7ed-42c0-9a9b-87969dbf4c02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:22.380182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1217/2463] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:21 2022] DELETE /identity/v3/projects/65cf0f36eb214b9b90daf93dd8d587ac => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:22.389497 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4b809-69b8-466c-8b29-5270cf748eec None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:22.389926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4b809-69b8-466c-8b29-5270cf748eec None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:22.390338 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4b809-69b8-466c-8b29-5270cf748eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:22.417779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-91d4b809-69b8-466c-8b29-5270cf748eec None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:22.419022 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-91d4b809-69b8-466c-8b29-5270cf748eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:22.439077 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-f3b09214-010a-42d3-885d-23d8f6f236b7 admin admin] The token cache is being invalidate because project 65d0e5c61cc54ec48d96c75060928e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:22.439398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-f3b09214-010a-42d3-885d-23d8f6f236b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:22.441111 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-f3b09214-010a-42d3-885d-23d8f6f236b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:22.442684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1247/2464] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:21 2022] DELETE /identity/v3/projects/65d0e5c61cc54ec48d96c75060928e88 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:19:22.447775 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b09214-010a-42d3-885d-23d8f6f236b7 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:22.450898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f3b09214-010a-42d3-885d-23d8f6f236b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:22.491646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:22.499861 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1218/2465] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:22.501554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d405fe9-b111-4bd4-ab1f-311901a7c115 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:22.501844 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d405fe9-b111-4bd4-ab1f-311901a7c115 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:22.502196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d405fe9-b111-4bd4-ab1f-311901a7c115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:22.523761 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-0d405fe9-b111-4bd4-ab1f-311901a7c115 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:22.524936 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-0d405fe9-b111-4bd4-ab1f-311901a7c115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:22.526843 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b09214-010a-42d3-885d-23d8f6f236b7 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:22.531611 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543cb9de-6ef2-4f42-a503-9f131bebdbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:22.532554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cb9de-6ef2-4f42-a503-9f131bebdbe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:22.532826 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cb9de-6ef2-4f42-a503-9f131bebdbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:22.533089 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cb9de-6ef2-4f42-a503-9f131bebdbe6 admin admin] PATH_INFO: `/v3/users/8b1a9ad4d9114098b14698039c2d445e` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:22.534289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543cb9de-6ef2-4f42-a503-9f131bebdbe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b1a9ad4d9114098b14698039c2d445e)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:22.535049 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1219/2466] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:22.538409 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543cb9de-6ef2-4f42-a503-9f131bebdbe6 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:22.546200 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d405fe9-b111-4bd4-ab1f-311901a7c115 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:22.556720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d405fe9-b111-4bd4-ab1f-311901a7c115 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:22.561718 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84678bef-4404-4b2c-a031-7881b18af65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:22.562666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84678bef-4404-4b2c-a031-7881b18af65e admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:22.562946 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84678bef-4404-4b2c-a031-7881b18af65e admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:22.563230 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84678bef-4404-4b2c-a031-7881b18af65e admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:22.563958 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84678bef-4404-4b2c-a031-7881b18af65e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:19:22.564233 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84678bef-4404-4b2c-a031-7881b18af65e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:22.565909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84678bef-4404-4b2c-a031-7881b18af65e admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:22.578094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1220/2467] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:19:22.582206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84678bef-4404-4b2c-a031-7881b18af65e admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:22.592017 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84678bef-4404-4b2c-a031-7881b18af65e admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:22.597345 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6379a-d59f-4ff3-afa4-4529de174b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:22.598237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6379a-d59f-4ff3-afa4-4529de174b0d admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:22.598237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6379a-d59f-4ff3-afa4-4529de174b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:22.598574 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6379a-d59f-4ff3-afa4-4529de174b0d admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:22.599208 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6379a-d59f-4ff3-afa4-4529de174b0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:22.601943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6379a-d59f-4ff3-afa4-4529de174b0d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:22.795652 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1221/2468] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:19:22 2022] POST /identity/v3/projects => generated 359 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:19:22.800022 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6379a-d59f-4ff3-afa4-4529de174b0d admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:22.863426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6379a-d59f-4ff3-afa4-4529de174b0d admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:22.868128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54afde-6cdf-48ce-8662-579d3d982e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:22.868757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54afde-6cdf-48ce-8662-579d3d982e19 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:22.869031 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54afde-6cdf-48ce-8662-579d3d982e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:22.869299 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54afde-6cdf-48ce-8662-579d3d982e19 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:22.869920 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54afde-6cdf-48ce-8662-579d3d982e19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:22.871855 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54afde-6cdf-48ce-8662-579d3d982e19 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:22.906507 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-543cb9de-6ef2-4f42-a503-9f131bebdbe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b1a9ad4d9114098b14698039c2d445e'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:22.907496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-543cb9de-6ef2-4f42-a503-9f131bebdbe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b1a9ad4d9114098b14698039c2d445e'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:22.916326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-543cb9de-6ef2-4f42-a503-9f131bebdbe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b1a9ad4d9114098b14698039c2d445e'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:22.980271 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1222/2469] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:19:22 2022] POST /identity/v3/users => generated 280 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:19:22.981995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54afde-6cdf-48ce-8662-579d3d982e19 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:22.984067 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6e54afde-6cdf-48ce-8662-579d3d982e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:23.000178 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54afde-6cdf-48ce-8662-579d3d982e19 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:23.005106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5698e39-4e11-4d83-b901-52529eb545b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:23.005783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5698e39-4e11-4d83-b901-52529eb545b0 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:23.006061 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5698e39-4e11-4d83-b901-52529eb545b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:23.006332 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5698e39-4e11-4d83-b901-52529eb545b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:23.006970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5698e39-4e11-4d83-b901-52529eb545b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:23.008660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5698e39-4e11-4d83-b901-52529eb545b0 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:23.019531 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1223/2470] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:23.023894 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5698e39-4e11-4d83-b901-52529eb545b0 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:23.035810 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5698e39-4e11-4d83-b901-52529eb545b0 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:23.041553 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c36c5-d749-4cfe-9093-4b91a72f9ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:23.042153 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c36c5-d749-4cfe-9093-4b91a72f9ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:23.042738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c36c5-d749-4cfe-9093-4b91a72f9ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:23.043004 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c36c5-d749-4cfe-9093-4b91a72f9ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:23.043662 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c36c5-d749-4cfe-9093-4b91a72f9ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:23.045263 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c36c5-d749-4cfe-9093-4b91a72f9ef4 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:23.056058 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1224/2471] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:19:23.060090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c36c5-d749-4cfe-9093-4b91a72f9ef4 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:23.061120 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-543cb9de-6ef2-4f42-a503-9f131bebdbe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b1a9ad4d9114098b14698039c2d445e'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:23.075374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-543cb9de-6ef2-4f42-a503-9f131bebdbe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b1a9ad4d9114098b14698039c2d445e'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:23.075553 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c36c5-d749-4cfe-9093-4b91a72f9ef4 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:23.076596 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188004f-8e13-400e-820d-e5ea51675f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:23.077251 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188004f-8e13-400e-820d-e5ea51675f7c admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:23.077528 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188004f-8e13-400e-820d-e5ea51675f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:23.077796 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188004f-8e13-400e-820d-e5ea51675f7c admin admin] PATH_INFO: `/v3/projects/8681a52d9e5f4a67bb6a4cc93649eccc/users/b707cee58966452eb5e51e3d25d1df70/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:23.084958 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1248/2472] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:19:22 2022] DELETE /identity/v3/users/8b1a9ad4d9114098b14698039c2d445e => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:19:23.095680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188004f-8e13-400e-820d-e5ea51675f7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8681a52d9e5f4a67bb6a4cc93649eccc, user_id=b707cee58966452eb5e51e3d25d1df70, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:23.098474 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188004f-8e13-400e-820d-e5ea51675f7c admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:23.234438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1225/2473] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:23 2022] PUT /identity/v3/projects/8681a52d9e5f4a67bb6a4cc93649eccc/users/b707cee58966452eb5e51e3d25d1df70/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:19:23.247575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88dc11-e8aa-41c3-a74b-00221899561b None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:23.247900 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88dc11-e8aa-41c3-a74b-00221899561b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:23.248201 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88dc11-e8aa-41c3-a74b-00221899561b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:23.278692 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-bf88dc11-e8aa-41c3-a74b-00221899561b None None] MFA Rules not processed for user `b707cee58966452eb5e51e3d25d1df70`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:23.279924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-bf88dc11-e8aa-41c3-a74b-00221899561b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:23.341436 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:23.351809 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1249/2474] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:23 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:19:23.390942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188004f-8e13-400e-820d-e5ea51675f7c admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:23.392960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3188004f-8e13-400e-820d-e5ea51675f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:23.461200 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188004f-8e13-400e-820d-e5ea51675f7c admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:23.465097 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e691714-5d49-4aff-a3b5-1b76897ed55c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:23.465673 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e691714-5d49-4aff-a3b5-1b76897ed55c service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:23.465945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e691714-5d49-4aff-a3b5-1b76897ed55c service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:23.466213 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e691714-5d49-4aff-a3b5-1b76897ed55c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:23.475090 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e691714-5d49-4aff-a3b5-1b76897ed55c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:23.476592 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e691714-5d49-4aff-a3b5-1b76897ed55c service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:23.485541 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1226/2475] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:23 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:19:23.502086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88dc11-e8aa-41c3-a74b-00221899561b None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:23.538309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e691714-5d49-4aff-a3b5-1b76897ed55c service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:23.540742 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6e691714-5d49-4aff-a3b5-1b76897ed55c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:23.577929 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88dc11-e8aa-41c3-a74b-00221899561b None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:23.582511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9591e5-0518-4989-a234-d519ca3f6159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:23.583139 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9591e5-0518-4989-a234-d519ca3f6159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:23.583441 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9591e5-0518-4989-a234-d519ca3f6159 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:23.583687 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9591e5-0518-4989-a234-d519ca3f6159 admin admin] PATH_INFO: `/v3/users/b707cee58966452eb5e51e3d25d1df70` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:23.584884 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9591e5-0518-4989-a234-d519ca3f6159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b707cee58966452eb5e51e3d25d1df70)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:23.586812 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9591e5-0518-4989-a234-d519ca3f6159 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:23.623733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e691714-5d49-4aff-a3b5-1b76897ed55c service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:23.629223 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0dcc6-8c42-4707-9d1b-9454f3b89376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:23.629893 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0dcc6-8c42-4707-9d1b-9454f3b89376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:23.630191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0dcc6-8c42-4707-9d1b-9454f3b89376 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:23.630512 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0dcc6-8c42-4707-9d1b-9454f3b89376 admin admin] PATH_INFO: `/v3/projects/08c24e75698a4c30820b12cb1315d791` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:23.632535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0dcc6-8c42-4707-9d1b-9454f3b89376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08c24e75698a4c30820b12cb1315d791)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:23.634467 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0dcc6-8c42-4707-9d1b-9454f3b89376 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:23.689844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:23.811869 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7d9591e5-0518-4989-a234-d519ca3f6159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b707cee58966452eb5e51e3d25d1df70'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:23.812787 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7d9591e5-0518-4989-a234-d519ca3f6159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b707cee58966452eb5e51e3d25d1df70'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:23.814328 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-86c0dcc6-8c42-4707-9d1b-9454f3b89376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08c24e75698a4c30820b12cb1315d791', 'request_id': 'req-86c0dcc6-8c42-4707-9d1b-9454f3b89376'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:23.823206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7d9591e5-0518-4989-a234-d519ca3f6159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b707cee58966452eb5e51e3d25d1df70'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:23.893548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-86c0dcc6-8c42-4707-9d1b-9454f3b89376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08c24e75698a4c30820b12cb1315d791', 'request_id': 'req-86c0dcc6-8c42-4707-9d1b-9454f3b89376'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:23.899874 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-86c0dcc6-8c42-4707-9d1b-9454f3b89376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08c24e75698a4c30820b12cb1315d791', 'request_id': 'req-86c0dcc6-8c42-4707-9d1b-9454f3b89376'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:23.911560 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-86c0dcc6-8c42-4707-9d1b-9454f3b89376 admin admin] The token cache is being invalidate because project 08c24e75698a4c30820b12cb1315d791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:23.911844 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-86c0dcc6-8c42-4707-9d1b-9454f3b89376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:23.912525 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-86c0dcc6-8c42-4707-9d1b-9454f3b89376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:23.913999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1227/2476] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:23 2022] DELETE /identity/v3/projects/08c24e75698a4c30820b12cb1315d791 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:19:23.919099 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97964448-045e-49f9-942c-40314ae3298f None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:23.919421 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97964448-045e-49f9-942c-40314ae3298f None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:23.919836 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97964448-045e-49f9-942c-40314ae3298f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:23.941219 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-97964448-045e-49f9-942c-40314ae3298f None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:23.942501 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-97964448-045e-49f9-942c-40314ae3298f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:23.956760 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7d9591e5-0518-4989-a234-d519ca3f6159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b707cee58966452eb5e51e3d25d1df70'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:23.959906 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7d9591e5-0518-4989-a234-d519ca3f6159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b707cee58966452eb5e51e3d25d1df70'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:23.982827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1250/2477] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:19:23 2022] DELETE /identity/v3/users/b707cee58966452eb5e51e3d25d1df70 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:23.987258 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d033c-da27-4afd-9449-d96e208518a7 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:23.987562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d033c-da27-4afd-9449-d96e208518a7 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:23.987859 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d033c-da27-4afd-9449-d96e208518a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:24.009070 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-542d033c-da27-4afd-9449-d96e208518a7 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:24.010134 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1228/2478] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:19:24.010682 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-542d033c-da27-4afd-9449-d96e208518a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:24.018439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155dd85-d207-4a3b-952b-fc5ebd9eff8c None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:24.018752 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155dd85-d207-4a3b-952b-fc5ebd9eff8c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:24.019047 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155dd85-d207-4a3b-952b-fc5ebd9eff8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:24.039353 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-f155dd85-d207-4a3b-952b-fc5ebd9eff8c None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:24.040624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f155dd85-d207-4a3b-952b-fc5ebd9eff8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:24.044681 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:24.072649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1251/2479] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:19:24.082437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d033c-da27-4afd-9449-d96e208518a7 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:24.083836 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-542d033c-da27-4afd-9449-d96e208518a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:24.109238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1229/2480] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:24.113813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155dd85-d207-4a3b-952b-fc5ebd9eff8c None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:24.123468 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155dd85-d207-4a3b-952b-fc5ebd9eff8c None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:24.128116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608e489-52b0-4173-b773-761a2fc6fa3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:24.128944 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608e489-52b0-4173-b773-761a2fc6fa3f admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:24.129398 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608e489-52b0-4173-b773-761a2fc6fa3f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:24.130033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608e489-52b0-4173-b773-761a2fc6fa3f admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:24.130736 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608e489-52b0-4173-b773-761a2fc6fa3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:19:24.131034 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608e489-52b0-4173-b773-761a2fc6fa3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:24.133306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608e489-52b0-4173-b773-761a2fc6fa3f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:24.143634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d033c-da27-4afd-9449-d96e208518a7 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:24.144957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1230/2481] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:19:24.147699 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257fc0d-5d3a-4e60-83a0-b51b447de516 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:24.148590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257fc0d-5d3a-4e60-83a0-b51b447de516 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:24.148871 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257fc0d-5d3a-4e60-83a0-b51b447de516 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:24.149142 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257fc0d-5d3a-4e60-83a0-b51b447de516 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:24.149319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608e489-52b0-4173-b773-761a2fc6fa3f admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:24.159100 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608e489-52b0-4173-b773-761a2fc6fa3f admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:24.160401 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257fc0d-5d3a-4e60-83a0-b51b447de516 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:24.161868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257fc0d-5d3a-4e60-83a0-b51b447de516 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:24.164362 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980db1a1-20b6-48ec-b7d9-891b6165c5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:24.164950 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980db1a1-20b6-48ec-b7d9-891b6165c5f0 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:24.165222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980db1a1-20b6-48ec-b7d9-891b6165c5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:24.165489 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980db1a1-20b6-48ec-b7d9-891b6165c5f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:24.166105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980db1a1-20b6-48ec-b7d9-891b6165c5f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:24.167983 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980db1a1-20b6-48ec-b7d9-891b6165c5f0 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:24.171931 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1252/2482] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:19:24.181873 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:24.239919 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1231/2483] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:19:24 2022] POST /identity/v3/projects => generated 385 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:19:24.244660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257fc0d-5d3a-4e60-83a0-b51b447de516 service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:24.250117 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:24.255820 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257fc0d-5d3a-4e60-83a0-b51b447de516 service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:24.305082 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980db1a1-20b6-48ec-b7d9-891b6165c5f0 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:24.327054 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3d356-60c5-4576-82df-2bd1803b8700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:24.327760 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3d356-60c5-4576-82df-2bd1803b8700 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:24.328081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3d356-60c5-4576-82df-2bd1803b8700 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:24.328379 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3d356-60c5-4576-82df-2bd1803b8700 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:24.329017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3d356-60c5-4576-82df-2bd1803b8700 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:24.330921 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3d356-60c5-4576-82df-2bd1803b8700 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:24.361976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980db1a1-20b6-48ec-b7d9-891b6165c5f0 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:24.365776 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b9944-c0e8-41df-a08d-f14a5db2d66f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:24.367056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b9944-c0e8-41df-a08d-f14a5db2d66f service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:24.367371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b9944-c0e8-41df-a08d-f14a5db2d66f service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:24.367640 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b9944-c0e8-41df-a08d-f14a5db2d66f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:24.369612 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-0d6b9944-c0e8-41df-a08d-f14a5db2d66f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:24.388264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b9944-c0e8-41df-a08d-f14a5db2d66f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:24.388264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b9944-c0e8-41df-a08d-f14a5db2d66f service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:24.399756 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1232/2484] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:19:24.419968 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1253/2485] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:19:24 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:19:24.421150 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b9944-c0e8-41df-a08d-f14a5db2d66f service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:24.433657 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b9944-c0e8-41df-a08d-f14a5db2d66f service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:24.441336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a11f6-cae8-4d76-9280-42a157dfd581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:24.445166 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a11f6-cae8-4d76-9280-42a157dfd581 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:24.445166 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a11f6-cae8-4d76-9280-42a157dfd581 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:24.445166 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a11f6-cae8-4d76-9280-42a157dfd581 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:24.445166 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a11f6-cae8-4d76-9280-42a157dfd581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:24.447207 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a11f6-cae8-4d76-9280-42a157dfd581 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:24.458423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1233/2486] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:24.463260 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3d356-60c5-4576-82df-2bd1803b8700 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:24.474065 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3d356-60c5-4576-82df-2bd1803b8700 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:24.480619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152a874-89ba-4058-aed2-228d03193dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:24.481358 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152a874-89ba-4058-aed2-228d03193dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:24.481791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152a874-89ba-4058-aed2-228d03193dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:24.482163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152a874-89ba-4058-aed2-228d03193dd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:24.482916 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152a874-89ba-4058-aed2-228d03193dd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:24.484655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152a874-89ba-4058-aed2-228d03193dd8 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:24.497721 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1254/2487] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:24.501936 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a11f6-cae8-4d76-9280-42a157dfd581 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:24.517165 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a11f6-cae8-4d76-9280-42a157dfd581 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:24.522083 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0f3aa-01d1-4ab8-99a2-4b132d0db9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:24.523370 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0f3aa-01d1-4ab8-99a2-4b132d0db9df admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:24.523716 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0f3aa-01d1-4ab8-99a2-4b132d0db9df admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:24.524053 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0f3aa-01d1-4ab8-99a2-4b132d0db9df admin admin] PATH_INFO: `/v3/projects/dc6f6ac628b54aaca55353f2fec3add8/users/5b77ece7d36446c3a184e5a43a8939f6/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:24.541022 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c0f3aa-01d1-4ab8-99a2-4b132d0db9df admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc6f6ac628b54aaca55353f2fec3add8, user_id=5b77ece7d36446c3a184e5a43a8939f6, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:24.543687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c0f3aa-01d1-4ab8-99a2-4b132d0db9df admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:24.600586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152a874-89ba-4058-aed2-228d03193dd8 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:24.603431 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8152a874-89ba-4058-aed2-228d03193dd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:24.618897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152a874-89ba-4058-aed2-228d03193dd8 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:24.631904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1234/2488] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:24 2022] PUT /identity/v3/projects/dc6f6ac628b54aaca55353f2fec3add8/users/5b77ece7d36446c3a184e5a43a8939f6/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:24.643085 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ae05a-134c-4e3e-88e9-0babc26e4ce8 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:24.643363 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ae05a-134c-4e3e-88e9-0babc26e4ce8 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:24.643649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ae05a-134c-4e3e-88e9-0babc26e4ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:24.675415 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-944ae05a-134c-4e3e-88e9-0babc26e4ce8 None None] MFA Rules not processed for user `5b77ece7d36446c3a184e5a43a8939f6`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:24.676653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-944ae05a-134c-4e3e-88e9-0babc26e4ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:24.697502 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef6c18-c486-435f-941e-1aaf5b00cbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:24.697794 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef6c18-c486-435f-941e-1aaf5b00cbf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:24.697912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef6c18-c486-435f-941e-1aaf5b00cbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:24.698160 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef6c18-c486-435f-941e-1aaf5b00cbf4 admin admin] PATH_INFO: `/v3/projects/8681a52d9e5f4a67bb6a4cc93649eccc` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:24.699368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef6c18-c486-435f-941e-1aaf5b00cbf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8681a52d9e5f4a67bb6a4cc93649eccc)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:24.701318 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef6c18-c486-435f-941e-1aaf5b00cbf4 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:24.749649 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1235/2489] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:19:24.754174 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ae05a-134c-4e3e-88e9-0babc26e4ce8 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:24.766233 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ae05a-134c-4e3e-88e9-0babc26e4ce8 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:24.771119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82160497-b0bd-4578-9de2-fbe360f34c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:24.782044 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82160497-b0bd-4578-9de2-fbe360f34c6f admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:24.782044 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82160497-b0bd-4578-9de2-fbe360f34c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:24.782044 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82160497-b0bd-4578-9de2-fbe360f34c6f admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:24.782044 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82160497-b0bd-4578-9de2-fbe360f34c6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:24.782044 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82160497-b0bd-4578-9de2-fbe360f34c6f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:24.813808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-91ef6c18-c486-435f-941e-1aaf5b00cbf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8681a52d9e5f4a67bb6a4cc93649eccc', 'request_id': 'req-91ef6c18-c486-435f-941e-1aaf5b00cbf4'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:24.820887 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-91ef6c18-c486-435f-941e-1aaf5b00cbf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8681a52d9e5f4a67bb6a4cc93649eccc', 'request_id': 'req-91ef6c18-c486-435f-941e-1aaf5b00cbf4'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:24.827671 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-91ef6c18-c486-435f-941e-1aaf5b00cbf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8681a52d9e5f4a67bb6a4cc93649eccc', 'request_id': 'req-91ef6c18-c486-435f-941e-1aaf5b00cbf4'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:24.847084 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1236/2490] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:19:24 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:19:24.848175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82160497-b0bd-4578-9de2-fbe360f34c6f admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:24.880816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:24.910964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-91ef6c18-c486-435f-941e-1aaf5b00cbf4 admin admin] The token cache is being invalidate because project 8681a52d9e5f4a67bb6a4cc93649eccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:24.911265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-91ef6c18-c486-435f-941e-1aaf5b00cbf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:24.911956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-91ef6c18-c486-435f-941e-1aaf5b00cbf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:24.914160 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1255/2491] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:24 2022] DELETE /identity/v3/projects/8681a52d9e5f4a67bb6a4cc93649eccc => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:24.925371 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a31d26-daeb-4d23-a786-2d7005dd84db None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:24.925699 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a31d26-daeb-4d23-a786-2d7005dd84db None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:24.925922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a31d26-daeb-4d23-a786-2d7005dd84db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:24.927651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82160497-b0bd-4578-9de2-fbe360f34c6f admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:24.936891 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1609e2c-ea06-441c-bb89-603f5827b160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:24.942844 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1609e2c-ea06-441c-bb89-603f5827b160 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:24.942844 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1609e2c-ea06-441c-bb89-603f5827b160 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:24.942844 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1609e2c-ea06-441c-bb89-603f5827b160 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:24.942844 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1609e2c-ea06-441c-bb89-603f5827b160 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:24.942844 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1609e2c-ea06-441c-bb89-603f5827b160 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:24.957403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-19a31d26-daeb-4d23-a786-2d7005dd84db None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:24.958605 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-19a31d26-daeb-4d23-a786-2d7005dd84db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:24.967725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1256/2492] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:24.976340 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82f67c-fe4b-4d57-8f53-cc76137c785b None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:24.977920 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82f67c-fe4b-4d57-8f53-cc76137c785b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:24.977920 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82f67c-fe4b-4d57-8f53-cc76137c785b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:24.987848 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:25.002170 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-eb82f67c-fe4b-4d57-8f53-cc76137c785b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:25.003465 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-eb82f67c-fe4b-4d57-8f53-cc76137c785b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:25.012784 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1257/2493] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:25.017349 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82f67c-fe4b-4d57-8f53-cc76137c785b None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:25.028265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82f67c-fe4b-4d57-8f53-cc76137c785b None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:25.033753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d02b41-af76-417e-86af-5e7b1f5733f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:25.034369 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d02b41-af76-417e-86af-5e7b1f5733f1 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:25.034645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d02b41-af76-417e-86af-5e7b1f5733f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:25.034920 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d02b41-af76-417e-86af-5e7b1f5733f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:25.035669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d02b41-af76-417e-86af-5e7b1f5733f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:19:25.035967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d02b41-af76-417e-86af-5e7b1f5733f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:25.037613 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d02b41-af76-417e-86af-5e7b1f5733f1 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:25.051349 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1258/2494] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:19:25.056058 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d02b41-af76-417e-86af-5e7b1f5733f1 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:25.066638 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1237/2495] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:19:24 2022] POST /identity/v3/users => generated 293 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:19:25.067722 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1609e2c-ea06-441c-bb89-603f5827b160 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:25.069139 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d02b41-af76-417e-86af-5e7b1f5733f1 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:25.070659 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b1609e2c-ea06-441c-bb89-603f5827b160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:25.074430 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88da745-a940-4c00-9709-581b0785db62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:25.075086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88da745-a940-4c00-9709-581b0785db62 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:25.075371 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88da745-a940-4c00-9709-581b0785db62 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:25.075645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88da745-a940-4c00-9709-581b0785db62 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:25.076348 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88da745-a940-4c00-9709-581b0785db62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:25.078540 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88da745-a940-4c00-9709-581b0785db62 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:25.083940 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1609e2c-ea06-441c-bb89-603f5827b160 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:25.090210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b99f6c-5a62-4295-8643-6f5c51ce2555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:25.091263 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b99f6c-5a62-4295-8643-6f5c51ce2555 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:25.091640 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b99f6c-5a62-4295-8643-6f5c51ce2555 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:25.101166 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b99f6c-5a62-4295-8643-6f5c51ce2555 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:25.101166 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b99f6c-5a62-4295-8643-6f5c51ce2555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:25.101166 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b99f6c-5a62-4295-8643-6f5c51ce2555 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:25.107188 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1238/2496] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:25.110846 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b99f6c-5a62-4295-8643-6f5c51ce2555 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:25.121691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b99f6c-5a62-4295-8643-6f5c51ce2555 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:25.126922 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e881e0-8981-410a-af1c-d510eabac6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:25.127951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e881e0-8981-410a-af1c-d510eabac6dd admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:25.128235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e881e0-8981-410a-af1c-d510eabac6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:25.128507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e881e0-8981-410a-af1c-d510eabac6dd admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:25.129143 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e881e0-8981-410a-af1c-d510eabac6dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:25.130741 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e881e0-8981-410a-af1c-d510eabac6dd admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:25.145491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1239/2497] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:25.149615 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e881e0-8981-410a-af1c-d510eabac6dd admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:25.160518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e881e0-8981-410a-af1c-d510eabac6dd admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:25.165882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eaa2c7-f06c-4434-8045-4aa1fada7044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:25.166622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eaa2c7-f06c-4434-8045-4aa1fada7044 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:25.166978 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eaa2c7-f06c-4434-8045-4aa1fada7044 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:25.167770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eaa2c7-f06c-4434-8045-4aa1fada7044 admin admin] PATH_INFO: `/v3/projects/6c79c4711ac04382b542d5b78c298ed5/users/1fb20b35287f4fe2bf5d25c9a5198cad/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:25.184392 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eaa2c7-f06c-4434-8045-4aa1fada7044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c79c4711ac04382b542d5b78c298ed5, user_id=1fb20b35287f4fe2bf5d25c9a5198cad, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:25.187080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eaa2c7-f06c-4434-8045-4aa1fada7044 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:25.251400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1259/2498] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:19:25 2022] POST /identity/v3/projects => generated 367 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:19:25.255479 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88da745-a940-4c00-9709-581b0785db62 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:25.321101 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88da745-a940-4c00-9709-581b0785db62 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:25.326687 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd27585-67c6-4848-96f2-11d13c4472c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:25.327290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd27585-67c6-4848-96f2-11d13c4472c3 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:25.327567 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd27585-67c6-4848-96f2-11d13c4472c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:25.327837 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd27585-67c6-4848-96f2-11d13c4472c3 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:25.328490 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd27585-67c6-4848-96f2-11d13c4472c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:25.330394 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd27585-67c6-4848-96f2-11d13c4472c3 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:25.364149 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1240/2499] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:25 2022] PUT /identity/v3/projects/6c79c4711ac04382b542d5b78c298ed5/users/1fb20b35287f4fe2bf5d25c9a5198cad/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:19:25.379712 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494a266-3973-4e39-a106-e9f5202eab08 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:25.380248 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494a266-3973-4e39-a106-e9f5202eab08 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:25.380711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494a266-3973-4e39-a106-e9f5202eab08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:25.417866 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-d494a266-3973-4e39-a106-e9f5202eab08 None None] MFA Rules not processed for user `1fb20b35287f4fe2bf5d25c9a5198cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:25.419117 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d494a266-3973-4e39-a106-e9f5202eab08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:25.433270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1260/2500] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:19:25 2022] POST /identity/v3/users => generated 284 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:19:25.439048 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd27585-67c6-4848-96f2-11d13c4472c3 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:25.468260 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd27585-67c6-4848-96f2-11d13c4472c3 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:25.540831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1241/2501] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:19:25.542688 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d494a266-3973-4e39-a106-e9f5202eab08 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:25.570841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec8a35-8316-4030-8a31-59b5950b2bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:25.572059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec8a35-8316-4030-8a31-59b5950b2bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:25.572666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec8a35-8316-4030-8a31-59b5950b2bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:25.573266 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec8a35-8316-4030-8a31-59b5950b2bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:25.574087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec8a35-8316-4030-8a31-59b5950b2bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:25.582616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec8a35-8316-4030-8a31-59b5950b2bf0 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:25.597994 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1261/2502] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:19:25.599035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec8a35-8316-4030-8a31-59b5950b2bf0 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:25.618347 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec8a35-8316-4030-8a31-59b5950b2bf0 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:25.633907 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa9ec4-2265-4773-9cb9-9a512de71ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:25.633907 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa9ec4-2265-4773-9cb9-9a512de71ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:25.633907 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa9ec4-2265-4773-9cb9-9a512de71ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:25.633907 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa9ec4-2265-4773-9cb9-9a512de71ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:25.634378 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa9ec4-2265-4773-9cb9-9a512de71ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:25.634378 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa9ec4-2265-4773-9cb9-9a512de71ef1 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:25.650152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1262/2503] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:25.652925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa9ec4-2265-4773-9cb9-9a512de71ef1 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:25.659822 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d494a266-3973-4e39-a106-e9f5202eab08 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:25.664988 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa9ec4-2265-4773-9cb9-9a512de71ef1 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:25.670446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10e617-9baa-4164-8c47-789134ac9737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:25.670968 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be735aed-0da7-47c4-b943-513fa2383556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:25.671660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10e617-9baa-4164-8c47-789134ac9737 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:25.672019 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10e617-9baa-4164-8c47-789134ac9737 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:25.672254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be735aed-0da7-47c4-b943-513fa2383556 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:25.672435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10e617-9baa-4164-8c47-789134ac9737 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:25.672867 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be735aed-0da7-47c4-b943-513fa2383556 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:25.673222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10e617-9baa-4164-8c47-789134ac9737 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:25.673479 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be735aed-0da7-47c4-b943-513fa2383556 admin admin] PATH_INFO: `/v3/projects/8d1d8bd83b2247bb9f59f1cf6d695fbf/users/b872b378c222451b97e3483bc8b3d766/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:25.675525 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10e617-9baa-4164-8c47-789134ac9737 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:25.680904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:25.691270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be735aed-0da7-47c4-b943-513fa2383556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d1d8bd83b2247bb9f59f1cf6d695fbf, user_id=b872b378c222451b97e3483bc8b3d766, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:25.694208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be735aed-0da7-47c4-b943-513fa2383556 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:25.793165 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1242/2504] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:19:25 2022] POST /identity/v3/projects => generated 387 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:19:25.797270 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10e617-9baa-4164-8c47-789134ac9737 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:25.823413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1263/2505] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:25 2022] PUT /identity/v3/projects/8d1d8bd83b2247bb9f59f1cf6d695fbf/users/b872b378c222451b97e3483bc8b3d766/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:25.833785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49790085-2bea-421c-a4cb-50d8077de59e None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:25.834067 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49790085-2bea-421c-a4cb-50d8077de59e None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:25.834341 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49790085-2bea-421c-a4cb-50d8077de59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:25.867896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:25.873583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-49790085-2bea-421c-a4cb-50d8077de59e None None] MFA Rules not processed for user `b872b378c222451b97e3483bc8b3d766`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:25.874820 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-49790085-2bea-421c-a4cb-50d8077de59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:25.876595 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10e617-9baa-4164-8c47-789134ac9737 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:25.942127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ee67e-2b45-4274-8c0e-b082a4fce15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:25.942912 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ee67e-2b45-4274-8c0e-b082a4fce15e admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:25.943360 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ee67e-2b45-4274-8c0e-b082a4fce15e admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:25.943657 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ee67e-2b45-4274-8c0e-b082a4fce15e admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:25.944973 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ee67e-2b45-4274-8c0e-b082a4fce15e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:25.947555 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ee67e-2b45-4274-8c0e-b082a4fce15e admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:25.968265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1264/2506] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:25.976796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49790085-2bea-421c-a4cb-50d8077de59e None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:25.978783 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-49790085-2bea-421c-a4cb-50d8077de59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:26.024644 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1243/2507] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:19:25 2022] POST /identity/v3/users => generated 300 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:19:26.029322 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ee67e-2b45-4274-8c0e-b082a4fce15e admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:26.050396 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ee67e-2b45-4274-8c0e-b082a4fce15e admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:26.057069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158ff9d-7cbc-442a-a2f9-956478093c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:26.057859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158ff9d-7cbc-442a-a2f9-956478093c4b admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:26.058214 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158ff9d-7cbc-442a-a2f9-956478093c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:26.058652 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158ff9d-7cbc-442a-a2f9-956478093c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:26.059383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158ff9d-7cbc-442a-a2f9-956478093c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:26.061354 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158ff9d-7cbc-442a-a2f9-956478093c4b admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:26.075816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1244/2508] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:19:26.095021 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158ff9d-7cbc-442a-a2f9-956478093c4b admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:26.112594 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49790085-2bea-421c-a4cb-50d8077de59e None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:26.118053 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158ff9d-7cbc-442a-a2f9-956478093c4b admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:26.123251 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a3ad8-119d-4ddc-a1e5-960b613e05b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:26.128313 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c55eb4-f064-4d68-9dd4-17881bec8a0e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4db08aa2e4ef4478a3909a876754814c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:26.128469 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a3ad8-119d-4ddc-a1e5-960b613e05b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:26.128639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c55eb4-f064-4d68-9dd4-17881bec8a0e service glance] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:26.128639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c55eb4-f064-4d68-9dd4-17881bec8a0e service glance] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:26.128639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c55eb4-f064-4d68-9dd4-17881bec8a0e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:26.128953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a3ad8-119d-4ddc-a1e5-960b613e05b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:26.128953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a3ad8-119d-4ddc-a1e5-960b613e05b0 admin admin] PATH_INFO: `/v3/projects/f6e50055e84140c894898b1cc976dcff/users/cd03ed4e6eb24bea85b865a7debec2d9/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:26.135666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c55eb4-f064-4d68-9dd4-17881bec8a0e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:26.137448 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c55eb4-f064-4d68-9dd4-17881bec8a0e service glance] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:26.148840 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1265/2509] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:19:26.149603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3a3ad8-119d-4ddc-a1e5-960b613e05b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6e50055e84140c894898b1cc976dcff, user_id=cd03ed4e6eb24bea85b865a7debec2d9, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:26.154082 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3a3ad8-119d-4ddc-a1e5-960b613e05b0 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:26.225738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1245/2510] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:26 2022] PUT /identity/v3/projects/f6e50055e84140c894898b1cc976dcff/users/cd03ed4e6eb24bea85b865a7debec2d9/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:26.230344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c55eb4-f064-4d68-9dd4-17881bec8a0e service glance] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:26.245460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c55eb4-f064-4d68-9dd4-17881bec8a0e service glance] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:26.324982 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eeb3e8-1bce-49a3-a4d0-307b698e25cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:26.325697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eeb3e8-1bce-49a3-a4d0-307b698e25cf admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:26.325990 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eeb3e8-1bce-49a3-a4d0-307b698e25cf admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:26.326250 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eeb3e8-1bce-49a3-a4d0-307b698e25cf admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:26.326945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eeb3e8-1bce-49a3-a4d0-307b698e25cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:26.328627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eeb3e8-1bce-49a3-a4d0-307b698e25cf admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:26.340794 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1266/2511] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:26.344668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a3ad8-119d-4ddc-a1e5-960b613e05b0 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:26.359792 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a3ad8-119d-4ddc-a1e5-960b613e05b0 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:26.364691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d8355-38f0-4950-be4c-a4230bb78137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:26.367321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d8355-38f0-4950-be4c-a4230bb78137 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:26.367321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d8355-38f0-4950-be4c-a4230bb78137 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:26.367321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d8355-38f0-4950-be4c-a4230bb78137 admin admin] PATH_INFO: `/v3/projects/f6e50055e84140c894898b1cc976dcff/users/cd03ed4e6eb24bea85b865a7debec2d9/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:26.369302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d8355-38f0-4950-be4c-a4230bb78137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6e50055e84140c894898b1cc976dcff, user_id=cd03ed4e6eb24bea85b865a7debec2d9, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:26.372826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d8355-38f0-4950-be4c-a4230bb78137 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:26.386747 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1246/2512] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:26 2022] PUT /identity/v3/projects/f6e50055e84140c894898b1cc976dcff/users/cd03ed4e6eb24bea85b865a7debec2d9/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:26.399367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb870c-042e-40b7-89c2-0bbff7fffd59 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:26.399656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb870c-042e-40b7-89c2-0bbff7fffd59 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:26.399901 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb870c-042e-40b7-89c2-0bbff7fffd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:26.442611 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-71eb870c-042e-40b7-89c2-0bbff7fffd59 None None] MFA Rules not processed for user `cd03ed4e6eb24bea85b865a7debec2d9`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:26.446092 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-71eb870c-042e-40b7-89c2-0bbff7fffd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:26.461893 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:26.555747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1267/2513] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:26 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:26.566950 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d8355-38f0-4950-be4c-a4230bb78137 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:26.569010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-324d8355-38f0-4950-be4c-a4230bb78137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:26.618679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:26.642707 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d8355-38f0-4950-be4c-a4230bb78137 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:26.646820 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd5c18-3de0-4c7b-b936-1ffb1b77cbfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:26.647537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd5c18-3de0-4c7b-b936-1ffb1b77cbfc service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:26.648338 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd5c18-3de0-4c7b-b936-1ffb1b77cbfc service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:26.648610 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd5c18-3de0-4c7b-b936-1ffb1b77cbfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:26.659244 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd5c18-3de0-4c7b-b936-1ffb1b77cbfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:26.660820 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd5c18-3de0-4c7b-b936-1ffb1b77cbfc service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:26.670953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1247/2514] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:26 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:19:28.185386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb870c-042e-40b7-89c2-0bbff7fffd59 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:28.190535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-71eb870c-042e-40b7-89c2-0bbff7fffd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:28.219700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb870c-042e-40b7-89c2-0bbff7fffd59 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:28.228240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631733a-a975-4e5d-9ff8-9dacadad7c17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:28.228240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631733a-a975-4e5d-9ff8-9dacadad7c17 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:28.228240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631733a-a975-4e5d-9ff8-9dacadad7c17 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:28.228240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631733a-a975-4e5d-9ff8-9dacadad7c17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:28.228702 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c631733a-a975-4e5d-9ff8-9dacadad7c17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:28.301443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd5c18-3de0-4c7b-b936-1ffb1b77cbfc service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:28.303076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-39bd5c18-3de0-4c7b-b936-1ffb1b77cbfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:28.310386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631733a-a975-4e5d-9ff8-9dacadad7c17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:28.311869 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631733a-a975-4e5d-9ff8-9dacadad7c17 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:28.322813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1268/2515] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:19:28.383707 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd5c18-3de0-4c7b-b936-1ffb1b77cbfc service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:28.395496 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834fed9-f629-4dd0-ab76-35a48cb4954a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:28.396217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834fed9-f629-4dd0-ab76-35a48cb4954a service placement] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:28.396501 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834fed9-f629-4dd0-ab76-35a48cb4954a service placement] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:28.396771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834fed9-f629-4dd0-ab76-35a48cb4954a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:28.399694 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-7834fed9-f629-4dd0-ab76-35a48cb4954a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:28.412461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834fed9-f629-4dd0-ab76-35a48cb4954a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:28.414640 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834fed9-f629-4dd0-ab76-35a48cb4954a service placement] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:28.427438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1248/2516] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:19:28.455385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631733a-a975-4e5d-9ff8-9dacadad7c17 service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:28.456964 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c631733a-a975-4e5d-9ff8-9dacadad7c17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:28.548743 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631733a-a975-4e5d-9ff8-9dacadad7c17 service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:28.553321 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072f087-2db9-4e22-8679-cb4f4c1f076b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:28.553995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072f087-2db9-4e22-8679-cb4f4c1f076b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:28.554271 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072f087-2db9-4e22-8679-cb4f4c1f076b admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:28.554545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072f087-2db9-4e22-8679-cb4f4c1f076b admin admin] PATH_INFO: `/v3/users/cc2f5c40aedf47dcb6374d64fcb04add` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:28.567773 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072f087-2db9-4e22-8679-cb4f4c1f076b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc2f5c40aedf47dcb6374d64fcb04add)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:28.571217 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072f087-2db9-4e22-8679-cb4f4c1f076b admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:28.741152 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:28.750048 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-a072f087-2db9-4e22-8679-cb4f4c1f076b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc2f5c40aedf47dcb6374d64fcb04add'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:28.750740 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-a072f087-2db9-4e22-8679-cb4f4c1f076b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc2f5c40aedf47dcb6374d64fcb04add'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:28.758464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-a072f087-2db9-4e22-8679-cb4f4c1f076b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc2f5c40aedf47dcb6374d64fcb04add'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:28.821207 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-a072f087-2db9-4e22-8679-cb4f4c1f076b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc2f5c40aedf47dcb6374d64fcb04add'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:28.821898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-a072f087-2db9-4e22-8679-cb4f4c1f076b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc2f5c40aedf47dcb6374d64fcb04add'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:28.840221 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1269/2517] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:19:28 2022] DELETE /identity/v3/users/cc2f5c40aedf47dcb6374d64fcb04add => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:29.086361 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834fed9-f629-4dd0-ab76-35a48cb4954a service placement] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:29.120113 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834fed9-f629-4dd0-ab76-35a48cb4954a service placement] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:29.213495 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64d794-b219-496e-9492-de33ba4992f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:29.213495 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64d794-b219-496e-9492-de33ba4992f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:29.213495 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64d794-b219-496e-9492-de33ba4992f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:29.213495 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64d794-b219-496e-9492-de33ba4992f3 admin admin] PATH_INFO: `/v3/projects/66de012537ed40e3bc233b80458bcfc4` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:29.214806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f64d794-b219-496e-9492-de33ba4992f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66de012537ed40e3bc233b80458bcfc4)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:29.216871 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f64d794-b219-496e-9492-de33ba4992f3 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:29.337275 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-3f64d794-b219-496e-9492-de33ba4992f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66de012537ed40e3bc233b80458bcfc4', 'request_id': 'req-3f64d794-b219-496e-9492-de33ba4992f3'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:29.422354 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-3f64d794-b219-496e-9492-de33ba4992f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66de012537ed40e3bc233b80458bcfc4', 'request_id': 'req-3f64d794-b219-496e-9492-de33ba4992f3'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:29.433219 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-3f64d794-b219-496e-9492-de33ba4992f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66de012537ed40e3bc233b80458bcfc4', 'request_id': 'req-3f64d794-b219-496e-9492-de33ba4992f3'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:29.436369 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:29.446665 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-3f64d794-b219-496e-9492-de33ba4992f3 admin admin] The token cache is being invalidate because project 66de012537ed40e3bc233b80458bcfc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:29.447227 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-3f64d794-b219-496e-9492-de33ba4992f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:29.447896 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-3f64d794-b219-496e-9492-de33ba4992f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:29.449402 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1249/2518] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:29 2022] DELETE /identity/v3/projects/66de012537ed40e3bc233b80458bcfc4 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:29.460737 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53073c-115a-40c5-8941-2650c22143f5 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:29.461251 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53073c-115a-40c5-8941-2650c22143f5 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:29.461348 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53073c-115a-40c5-8941-2650c22143f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:29.482221 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-ff53073c-115a-40c5-8941-2650c22143f5 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:29.483415 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-ff53073c-115a-40c5-8941-2650c22143f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:29.544871 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1270/2519] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:29.552694 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30158a95-535e-4b9f-84b0-210c5c4b3975 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:29.552978 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30158a95-535e-4b9f-84b0-210c5c4b3975 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:29.553252 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30158a95-535e-4b9f-84b0-210c5c4b3975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:29.575628 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-30158a95-535e-4b9f-84b0-210c5c4b3975 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:29.576917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-30158a95-535e-4b9f-84b0-210c5c4b3975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:29.585151 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1250/2520] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:29.589497 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53073c-115a-40c5-8941-2650c22143f5 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:29.599758 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53073c-115a-40c5-8941-2650c22143f5 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:29.605054 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7298f4-dff9-4a62-a45c-a3d235950f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:29.605651 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7298f4-dff9-4a62-a45c-a3d235950f68 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:29.605942 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7298f4-dff9-4a62-a45c-a3d235950f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:29.606322 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7298f4-dff9-4a62-a45c-a3d235950f68 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:29.606968 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7298f4-dff9-4a62-a45c-a3d235950f68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:19:29.607267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7298f4-dff9-4a62-a45c-a3d235950f68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:29.608893 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7298f4-dff9-4a62-a45c-a3d235950f68 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:29.621897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1271/2521] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:19:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:19:29.626293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30158a95-535e-4b9f-84b0-210c5c4b3975 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:29.636997 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30158a95-535e-4b9f-84b0-210c5c4b3975 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:29.641789 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2149f-1c06-46c1-9ad3-aec73fabcc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:29.642413 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2149f-1c06-46c1-9ad3-aec73fabcc67 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:29.642679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2149f-1c06-46c1-9ad3-aec73fabcc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:29.642945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2149f-1c06-46c1-9ad3-aec73fabcc67 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:29.643584 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2149f-1c06-46c1-9ad3-aec73fabcc67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:29.645610 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2149f-1c06-46c1-9ad3-aec73fabcc67 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:29.726056 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1251/2522] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:19:29 2022] POST /identity/v3/projects => generated 405 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:19:29.730352 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7298f4-dff9-4a62-a45c-a3d235950f68 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:29.740450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7298f4-dff9-4a62-a45c-a3d235950f68 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:29.803952 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d476453-089e-4af1-b8b9-1b84e2725239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:29.804537 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d476453-089e-4af1-b8b9-1b84e2725239 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:29.804808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d476453-089e-4af1-b8b9-1b84e2725239 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:29.805073 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d476453-089e-4af1-b8b9-1b84e2725239 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:29.805689 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d476453-089e-4af1-b8b9-1b84e2725239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:29.807616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d476453-089e-4af1-b8b9-1b84e2725239 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:29.919352 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1272/2523] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:19:29 2022] POST /identity/v3/users => generated 303 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:19:29.923867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2149f-1c06-46c1-9ad3-aec73fabcc67 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:29.934693 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2149f-1c06-46c1-9ad3-aec73fabcc67 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:29.939344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e22d97-e385-46f7-8f80-5fe646e4c226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:29.939968 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e22d97-e385-46f7-8f80-5fe646e4c226 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:29.940285 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e22d97-e385-46f7-8f80-5fe646e4c226 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:29.940552 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e22d97-e385-46f7-8f80-5fe646e4c226 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:29.941180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e22d97-e385-46f7-8f80-5fe646e4c226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:29.942806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e22d97-e385-46f7-8f80-5fe646e4c226 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:29.954454 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1252/2524] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:29.958660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d476453-089e-4af1-b8b9-1b84e2725239 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:29.969478 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d476453-089e-4af1-b8b9-1b84e2725239 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:29.974572 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9e6c1-cd80-4961-ae25-90bdf8696ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:29.975173 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9e6c1-cd80-4961-ae25-90bdf8696ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:29.975457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9e6c1-cd80-4961-ae25-90bdf8696ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:29.975730 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9e6c1-cd80-4961-ae25-90bdf8696ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:29.976604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9e6c1-cd80-4961-ae25-90bdf8696ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:29.978263 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9e6c1-cd80-4961-ae25-90bdf8696ae3 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:29.989787 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1273/2525] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:29.994707 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e22d97-e385-46f7-8f80-5fe646e4c226 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:30.005057 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e22d97-e385-46f7-8f80-5fe646e4c226 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:30.010543 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3d3a2-9ab6-489c-852b-bd33cabda8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:30.011181 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3d3a2-9ab6-489c-852b-bd33cabda8e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:30.011458 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3d3a2-9ab6-489c-852b-bd33cabda8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:30.011730 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3d3a2-9ab6-489c-852b-bd33cabda8e2 admin admin] PATH_INFO: `/v3/projects/ec9d5cf510cd4d69a45f53ec3d39e31b/users/32bc0fee994d4ed3a302188a1e1822ce/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:30.032896 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3d3a2-9ab6-489c-852b-bd33cabda8e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec9d5cf510cd4d69a45f53ec3d39e31b, user_id=32bc0fee994d4ed3a302188a1e1822ce, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:30.035732 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3d3a2-9ab6-489c-852b-bd33cabda8e2 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:30.119009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1253/2526] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:29 2022] PUT /identity/v3/projects/ec9d5cf510cd4d69a45f53ec3d39e31b/users/32bc0fee994d4ed3a302188a1e1822ce/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:30.139468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208de853-3e59-4abb-9016-69cb909a4f72 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:30.139468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208de853-3e59-4abb-9016-69cb909a4f72 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:30.139468 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208de853-3e59-4abb-9016-69cb909a4f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:30.158453 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-208de853-3e59-4abb-9016-69cb909a4f72 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:30.159666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-208de853-3e59-4abb-9016-69cb909a4f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:30.178087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:30.229620 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1274/2527] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:30.240206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3d3a2-9ab6-489c-852b-bd33cabda8e2 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:30.243879 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-46a3d3a2-9ab6-489c-852b-bd33cabda8e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:30.311345 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3d3a2-9ab6-489c-852b-bd33cabda8e2 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:30.315060 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72858e32-864d-4469-b298-b9f97b2e0ea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:30.315673 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72858e32-864d-4469-b298-b9f97b2e0ea3 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:30.315954 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72858e32-864d-4469-b298-b9f97b2e0ea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:30.316323 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72858e32-864d-4469-b298-b9f97b2e0ea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:30.326525 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72858e32-864d-4469-b298-b9f97b2e0ea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:30.331250 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72858e32-864d-4469-b298-b9f97b2e0ea3 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:30.341672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1254/2528] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:19:31.977055 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0482e-8650-45f9-8d22-73fe035734a0 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:31.977918 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0482e-8650-45f9-8d22-73fe035734a0 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:31.978241 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0482e-8650-45f9-8d22-73fe035734a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:32.019179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-5af0482e-8650-45f9-8d22-73fe035734a0 None None] MFA Rules not processed for user `32bc0fee994d4ed3a302188a1e1822ce`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:32.020476 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-5af0482e-8650-45f9-8d22-73fe035734a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:32.095419 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1275/2529] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:31 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:32.099952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72858e32-864d-4469-b298-b9f97b2e0ea3 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:32.111435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72858e32-864d-4469-b298-b9f97b2e0ea3 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:32.117074 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520eab35-cd16-4e28-ab79-41ec56e463f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:32.117718 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520eab35-cd16-4e28-ab79-41ec56e463f1 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:32.118001 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520eab35-cd16-4e28-ab79-41ec56e463f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:32.118275 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520eab35-cd16-4e28-ab79-41ec56e463f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:32.118907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520eab35-cd16-4e28-ab79-41ec56e463f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:32.120962 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520eab35-cd16-4e28-ab79-41ec56e463f1 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:32.232294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1255/2530] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:19:32 2022] POST /identity/v3/projects => generated 403 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:19:32.233425 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0482e-8650-45f9-8d22-73fe035734a0 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:32.244630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0482e-8650-45f9-8d22-73fe035734a0 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:32.310612 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184a70c-0522-4561-a4f2-f698827ca315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:32.311233 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184a70c-0522-4561-a4f2-f698827ca315 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:32.311506 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184a70c-0522-4561-a4f2-f698827ca315 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:32.311773 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184a70c-0522-4561-a4f2-f698827ca315 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:32.312442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8184a70c-0522-4561-a4f2-f698827ca315 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:32.314382 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8184a70c-0522-4561-a4f2-f698827ca315 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:32.437495 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1276/2531] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:19:32 2022] POST /identity/v3/users => generated 308 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:19:32.441479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520eab35-cd16-4e28-ab79-41ec56e463f1 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:32.453167 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520eab35-cd16-4e28-ab79-41ec56e463f1 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:32.458115 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722287fa-6b07-4306-bd6d-3001ab8cf39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:32.458736 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722287fa-6b07-4306-bd6d-3001ab8cf39b admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:32.459181 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722287fa-6b07-4306-bd6d-3001ab8cf39b admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:32.459484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722287fa-6b07-4306-bd6d-3001ab8cf39b admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:32.460128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722287fa-6b07-4306-bd6d-3001ab8cf39b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:32.462481 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722287fa-6b07-4306-bd6d-3001ab8cf39b admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:32.474356 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1256/2532] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:32.482742 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184a70c-0522-4561-a4f2-f698827ca315 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:32.493085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184a70c-0522-4561-a4f2-f698827ca315 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:32.497764 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe3746-4524-41cc-bb35-ddeaea8a90e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:32.498410 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe3746-4524-41cc-bb35-ddeaea8a90e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:32.498693 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe3746-4524-41cc-bb35-ddeaea8a90e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:32.498962 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe3746-4524-41cc-bb35-ddeaea8a90e8 admin admin] PATH_INFO: `/v3/projects/ea68b226c2ae44ed8b1ff88c2382c982/users/e1e1acd88e304101bd97f86bd0e5d1bc/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:32.514036 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe3746-4524-41cc-bb35-ddeaea8a90e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea68b226c2ae44ed8b1ff88c2382c982, user_id=e1e1acd88e304101bd97f86bd0e5d1bc, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:32.516730 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe3746-4524-41cc-bb35-ddeaea8a90e8 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:32.593472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1277/2533] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:32 2022] PUT /identity/v3/projects/ea68b226c2ae44ed8b1ff88c2382c982/users/e1e1acd88e304101bd97f86bd0e5d1bc/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:32.597616 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722287fa-6b07-4306-bd6d-3001ab8cf39b admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:32.608006 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722287fa-6b07-4306-bd6d-3001ab8cf39b admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:32.666371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357745b5-3869-4790-8687-7a15ee6825a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:32.666996 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357745b5-3869-4790-8687-7a15ee6825a3 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:32.667275 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357745b5-3869-4790-8687-7a15ee6825a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:32.667572 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357745b5-3869-4790-8687-7a15ee6825a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:32.668237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357745b5-3869-4790-8687-7a15ee6825a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:32.669845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357745b5-3869-4790-8687-7a15ee6825a3 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:32.672476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:32.680609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1257/2534] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:32.684855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe3746-4524-41cc-bb35-ddeaea8a90e8 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:32.695958 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe3746-4524-41cc-bb35-ddeaea8a90e8 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:32.701034 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f1c0d-c564-40bf-bc1c-f62c21b2d890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:32.701719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f1c0d-c564-40bf-bc1c-f62c21b2d890 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:32.702039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f1c0d-c564-40bf-bc1c-f62c21b2d890 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:32.702334 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f1c0d-c564-40bf-bc1c-f62c21b2d890 admin admin] PATH_INFO: `/v3/projects/ea68b226c2ae44ed8b1ff88c2382c982/users/e1e1acd88e304101bd97f86bd0e5d1bc/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:32.704317 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f1c0d-c564-40bf-bc1c-f62c21b2d890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea68b226c2ae44ed8b1ff88c2382c982, user_id=e1e1acd88e304101bd97f86bd0e5d1bc, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:32.706993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f1c0d-c564-40bf-bc1c-f62c21b2d890 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:32.720019 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1278/2535] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:32 2022] PUT /identity/v3/projects/ea68b226c2ae44ed8b1ff88c2382c982/users/e1e1acd88e304101bd97f86bd0e5d1bc/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:33.647999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357745b5-3869-4790-8687-7a15ee6825a3 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:33.650988 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-357745b5-3869-4790-8687-7a15ee6825a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:33.720302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357745b5-3869-4790-8687-7a15ee6825a3 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:33.769150 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a0301-c108-492f-aae0-701378974cbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:33.769818 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a0301-c108-492f-aae0-701378974cbc service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:33.770074 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a0301-c108-492f-aae0-701378974cbc service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:33.770351 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a0301-c108-492f-aae0-701378974cbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:33.772544 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-e41a0301-c108-492f-aae0-701378974cbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:33.836849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a0301-c108-492f-aae0-701378974cbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:33.838312 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a0301-c108-492f-aae0-701378974cbc service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:33.850512 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1258/2536] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:19:34.221811 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b1f0e-3121-4d97-8298-d72288046e98 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:34.222397 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b1f0e-3121-4d97-8298-d72288046e98 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:34.222810 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b1f0e-3121-4d97-8298-d72288046e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:34.231031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:34.259458 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-391b1f0e-3121-4d97-8298-d72288046e98 None None] MFA Rules not processed for user `e1e1acd88e304101bd97f86bd0e5d1bc`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:34.263487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-391b1f0e-3121-4d97-8298-d72288046e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:34.356389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1279/2537] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:34 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:35.964966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a0301-c108-492f-aae0-701378974cbc service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:36.043433 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a0301-c108-492f-aae0-701378974cbc service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:36.048067 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f453910-3eb5-4598-9ecd-9eb9284b2996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:36.053806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f453910-3eb5-4598-9ecd-9eb9284b2996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:36.053806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f453910-3eb5-4598-9ecd-9eb9284b2996 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:36.053806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f453910-3eb5-4598-9ecd-9eb9284b2996 admin admin] PATH_INFO: `/v3/users/32bc0fee994d4ed3a302188a1e1822ce` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:36.053806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f453910-3eb5-4598-9ecd-9eb9284b2996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32bc0fee994d4ed3a302188a1e1822ce)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:36.053806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f453910-3eb5-4598-9ecd-9eb9284b2996 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:36.063912 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:36.269258 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-9f453910-3eb5-4598-9ecd-9eb9284b2996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32bc0fee994d4ed3a302188a1e1822ce'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:36.276327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-9f453910-3eb5-4598-9ecd-9eb9284b2996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32bc0fee994d4ed3a302188a1e1822ce'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:36.277004 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-9f453910-3eb5-4598-9ecd-9eb9284b2996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32bc0fee994d4ed3a302188a1e1822ce'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:36.295796 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-9f453910-3eb5-4598-9ecd-9eb9284b2996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32bc0fee994d4ed3a302188a1e1822ce'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:36.296579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-9f453910-3eb5-4598-9ecd-9eb9284b2996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32bc0fee994d4ed3a302188a1e1822ce'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:36.382025 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1259/2538] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:19:35 2022] DELETE /identity/v3/users/32bc0fee994d4ed3a302188a1e1822ce => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:36.669272 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b1f0e-3121-4d97-8298-d72288046e98 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:36.669272 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b1f0e-3121-4d97-8298-d72288046e98 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:36.731073 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1273d6-3805-4bb4-bd31-8edc848402cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:36.731938 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1273d6-3805-4bb4-bd31-8edc848402cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:36.732246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1273d6-3805-4bb4-bd31-8edc848402cd admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:36.732522 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1273d6-3805-4bb4-bd31-8edc848402cd admin admin] PATH_INFO: `/v3/projects/ec9d5cf510cd4d69a45f53ec3d39e31b` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:36.733671 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1273d6-3805-4bb4-bd31-8edc848402cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec9d5cf510cd4d69a45f53ec3d39e31b)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:36.735565 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1273d6-3805-4bb4-bd31-8edc848402cd admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:36.906003 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6f1273d6-3805-4bb4-bd31-8edc848402cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec9d5cf510cd4d69a45f53ec3d39e31b', 'request_id': 'req-6f1273d6-3805-4bb4-bd31-8edc848402cd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:36.914661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6f1273d6-3805-4bb4-bd31-8edc848402cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec9d5cf510cd4d69a45f53ec3d39e31b', 'request_id': 'req-6f1273d6-3805-4bb4-bd31-8edc848402cd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:36.920996 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6f1273d6-3805-4bb4-bd31-8edc848402cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec9d5cf510cd4d69a45f53ec3d39e31b', 'request_id': 'req-6f1273d6-3805-4bb4-bd31-8edc848402cd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:37.000036 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6f1273d6-3805-4bb4-bd31-8edc848402cd admin admin] The token cache is being invalidate because project ec9d5cf510cd4d69a45f53ec3d39e31b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:37.005028 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6f1273d6-3805-4bb4-bd31-8edc848402cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:37.005028 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6f1273d6-3805-4bb4-bd31-8edc848402cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:37.005028 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1280/2539] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:36 2022] DELETE /identity/v3/projects/ec9d5cf510cd4d69a45f53ec3d39e31b => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:37.006771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f453910-3eb5-4598-9ecd-9eb9284b2996 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:37.008979 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-9f453910-3eb5-4598-9ecd-9eb9284b2996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:37.023729 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f453910-3eb5-4598-9ecd-9eb9284b2996 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:37.092541 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c60f2-9e69-44f6-932c-03b2a00ced90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:37.093194 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c60f2-9e69-44f6-932c-03b2a00ced90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:37.093502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c60f2-9e69-44f6-932c-03b2a00ced90 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:37.093769 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c60f2-9e69-44f6-932c-03b2a00ced90 admin admin] PATH_INFO: `/v3/users/e1e1acd88e304101bd97f86bd0e5d1bc` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:37.094905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c60f2-9e69-44f6-932c-03b2a00ced90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1e1acd88e304101bd97f86bd0e5d1bc)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:37.096878 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c60f2-9e69-44f6-932c-03b2a00ced90 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:37.350555 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-cb5c60f2-9e69-44f6-932c-03b2a00ced90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1e1acd88e304101bd97f86bd0e5d1bc'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:37.358099 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-cb5c60f2-9e69-44f6-932c-03b2a00ced90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1e1acd88e304101bd97f86bd0e5d1bc'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:37.358757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-cb5c60f2-9e69-44f6-932c-03b2a00ced90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1e1acd88e304101bd97f86bd0e5d1bc'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:37.370532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-cb5c60f2-9e69-44f6-932c-03b2a00ced90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1e1acd88e304101bd97f86bd0e5d1bc'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:37.371166 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-cb5c60f2-9e69-44f6-932c-03b2a00ced90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1e1acd88e304101bd97f86bd0e5d1bc'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:37.487485 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1260/2540] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:19:37 2022] DELETE /identity/v3/users/e1e1acd88e304101bd97f86bd0e5d1bc => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:37.738650 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1273d6-3805-4bb4-bd31-8edc848402cd admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:37.743257 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6f1273d6-3805-4bb4-bd31-8edc848402cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:37.755345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1273d6-3805-4bb4-bd31-8edc848402cd admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:37.783562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:37.831897 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263a2c9-f19e-402c-9043-38d086787f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:37.837035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263a2c9-f19e-402c-9043-38d086787f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:37.837035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263a2c9-f19e-402c-9043-38d086787f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:37.837035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263a2c9-f19e-402c-9043-38d086787f02 admin admin] PATH_INFO: `/v3/projects/ea68b226c2ae44ed8b1ff88c2382c982` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:37.837035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263a2c9-f19e-402c-9043-38d086787f02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea68b226c2ae44ed8b1ff88c2382c982)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:37.837035 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263a2c9-f19e-402c-9043-38d086787f02 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:37.997587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0263a2c9-f19e-402c-9043-38d086787f02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea68b226c2ae44ed8b1ff88c2382c982', 'request_id': 'req-0263a2c9-f19e-402c-9043-38d086787f02'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:38.004023 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0263a2c9-f19e-402c-9043-38d086787f02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea68b226c2ae44ed8b1ff88c2382c982', 'request_id': 'req-0263a2c9-f19e-402c-9043-38d086787f02'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:38.009974 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0263a2c9-f19e-402c-9043-38d086787f02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea68b226c2ae44ed8b1ff88c2382c982', 'request_id': 'req-0263a2c9-f19e-402c-9043-38d086787f02'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:38.107267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0263a2c9-f19e-402c-9043-38d086787f02 admin admin] The token cache is being invalidate because project ea68b226c2ae44ed8b1ff88c2382c982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:38.107690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0263a2c9-f19e-402c-9043-38d086787f02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:38.108229 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0263a2c9-f19e-402c-9043-38d086787f02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:38.109739 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1281/2541] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:37 2022] DELETE /identity/v3/projects/ea68b226c2ae44ed8b1ff88c2382c982 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:38.115836 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cd540-7051-4bfe-aa11-a314436b7453 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:38.116446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cd540-7051-4bfe-aa11-a314436b7453 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:38.116687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cd540-7051-4bfe-aa11-a314436b7453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:38.133724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:38.139956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-153cd540-7051-4bfe-aa11-a314436b7453 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:38.141231 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-153cd540-7051-4bfe-aa11-a314436b7453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:38.208548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1261/2542] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:38.211298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a4b59-b299-44c2-94cc-bafbe654a8ca None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:38.211606 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a4b59-b299-44c2-94cc-bafbe654a8ca None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:38.211882 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a4b59-b299-44c2-94cc-bafbe654a8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:38.232704 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-494a4b59-b299-44c2-94cc-bafbe654a8ca None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:38.233913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-494a4b59-b299-44c2-94cc-bafbe654a8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:38.242634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1282/2543] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:19:38.247104 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cd540-7051-4bfe-aa11-a314436b7453 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:38.256998 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cd540-7051-4bfe-aa11-a314436b7453 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:38.262223 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bfa03-3a87-48f5-9620-9ea0e712f46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:38.262815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bfa03-3a87-48f5-9620-9ea0e712f46b admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:38.263088 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bfa03-3a87-48f5-9620-9ea0e712f46b admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:38.263352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bfa03-3a87-48f5-9620-9ea0e712f46b admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:38.263986 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bfa03-3a87-48f5-9620-9ea0e712f46b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:19:38.264515 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bfa03-3a87-48f5-9620-9ea0e712f46b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:38.270444 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bfa03-3a87-48f5-9620-9ea0e712f46b admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:38.278971 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1262/2544] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:19:38.283555 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a4b59-b299-44c2-94cc-bafbe654a8ca None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:38.294980 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a4b59-b299-44c2-94cc-bafbe654a8ca None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:38.299827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8cb39-33da-4e77-8483-364126a3f08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:38.300467 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8cb39-33da-4e77-8483-364126a3f08a admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:38.300748 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8cb39-33da-4e77-8483-364126a3f08a admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:38.301184 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8cb39-33da-4e77-8483-364126a3f08a admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:38.301825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8cb39-33da-4e77-8483-364126a3f08a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:38.303719 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8cb39-33da-4e77-8483-364126a3f08a admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:38.398062 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1283/2545] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:19:38 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:19:38.402389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bfa03-3a87-48f5-9620-9ea0e712f46b admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:38.416478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bfa03-3a87-48f5-9620-9ea0e712f46b admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:38.468579 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8cb39-33da-4e77-8483-364126a3f08a admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:38.471869 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-46c8cb39-33da-4e77-8483-364126a3f08a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:38.477682 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57888c-2a9b-4308-9968-39ac77967128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:38.478802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57888c-2a9b-4308-9968-39ac77967128 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:38.479146 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57888c-2a9b-4308-9968-39ac77967128 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:38.479461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57888c-2a9b-4308-9968-39ac77967128 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:38.480185 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57888c-2a9b-4308-9968-39ac77967128 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:38.484448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57888c-2a9b-4308-9968-39ac77967128 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:38.544881 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8cb39-33da-4e77-8483-364126a3f08a admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:38.549635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde7c42-11dd-4610-b9ed-54e3b56bbe0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:38.550268 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde7c42-11dd-4610-b9ed-54e3b56bbe0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:38.550545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde7c42-11dd-4610-b9ed-54e3b56bbe0a admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:38.550815 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde7c42-11dd-4610-b9ed-54e3b56bbe0a admin admin] PATH_INFO: `/v3/users/5b77ece7d36446c3a184e5a43a8939f6` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:38.551950 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edde7c42-11dd-4610-b9ed-54e3b56bbe0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b77ece7d36446c3a184e5a43a8939f6)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:38.553948 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edde7c42-11dd-4610-b9ed-54e3b56bbe0a admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:38.566544 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1263/2546] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:19:38 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:19:38.567944 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57888c-2a9b-4308-9968-39ac77967128 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:38.577128 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:38.578806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57888c-2a9b-4308-9968-39ac77967128 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:38.583518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0baa6c-bcce-4c0a-9028-5759cf4fd726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:38.584113 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0baa6c-bcce-4c0a-9028-5759cf4fd726 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:38.584448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0baa6c-bcce-4c0a-9028-5759cf4fd726 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:38.584690 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0baa6c-bcce-4c0a-9028-5759cf4fd726 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:38.585361 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0baa6c-bcce-4c0a-9028-5759cf4fd726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:38.587007 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0baa6c-bcce-4c0a-9028-5759cf4fd726 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:38.597815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1264/2547] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:19:38.602525 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0baa6c-bcce-4c0a-9028-5759cf4fd726 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:38.613215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0baa6c-bcce-4c0a-9028-5759cf4fd726 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:38.619389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddf6a9-ce71-4b5c-bc94-563b0d812d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:38.620022 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddf6a9-ce71-4b5c-bc94-563b0d812d51 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:38.620409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddf6a9-ce71-4b5c-bc94-563b0d812d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:38.623821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddf6a9-ce71-4b5c-bc94-563b0d812d51 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:38.623821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cddf6a9-ce71-4b5c-bc94-563b0d812d51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:38.623821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cddf6a9-ce71-4b5c-bc94-563b0d812d51 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:38.634030 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1265/2548] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:19:38.638338 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddf6a9-ce71-4b5c-bc94-563b0d812d51 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:38.648412 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddf6a9-ce71-4b5c-bc94-563b0d812d51 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:38.653606 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac805c3-0402-4e0c-b2a1-e964ffff0bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:38.654247 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac805c3-0402-4e0c-b2a1-e964ffff0bca admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:38.654523 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac805c3-0402-4e0c-b2a1-e964ffff0bca admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:38.654794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac805c3-0402-4e0c-b2a1-e964ffff0bca admin admin] PATH_INFO: `/v3/projects/55e6d241d1034523bcface43ee8b7b94/users/8ecd044ac97747dc8957d62f89a10a6f/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:38.670427 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac805c3-0402-4e0c-b2a1-e964ffff0bca admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e6d241d1034523bcface43ee8b7b94, user_id=8ecd044ac97747dc8957d62f89a10a6f, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:38.673129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac805c3-0402-4e0c-b2a1-e964ffff0bca admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:38.850594 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1266/2549] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:38 2022] PUT /identity/v3/projects/55e6d241d1034523bcface43ee8b7b94/users/8ecd044ac97747dc8957d62f89a10a6f/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:19:38.861930 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139440c1-717a-4901-a939-24d45893da0a None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:38.862189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139440c1-717a-4901-a939-24d45893da0a None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:38.862460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139440c1-717a-4901-a939-24d45893da0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:38.868273 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-edde7c42-11dd-4610-b9ed-54e3b56bbe0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b77ece7d36446c3a184e5a43a8939f6'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:38.868932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-edde7c42-11dd-4610-b9ed-54e3b56bbe0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b77ece7d36446c3a184e5a43a8939f6'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:38.876579 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-edde7c42-11dd-4610-b9ed-54e3b56bbe0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b77ece7d36446c3a184e5a43a8939f6'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:38.891688 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-139440c1-717a-4901-a939-24d45893da0a None None] MFA Rules not processed for user `8ecd044ac97747dc8957d62f89a10a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:38.892927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-139440c1-717a-4901-a939-24d45893da0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:38.920574 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:38.961712 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1267/2550] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:19:38.968165 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139440c1-717a-4901-a939-24d45893da0a None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:38.969538 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-139440c1-717a-4901-a939-24d45893da0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:38.987730 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-edde7c42-11dd-4610-b9ed-54e3b56bbe0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b77ece7d36446c3a184e5a43a8939f6'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:38.988584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-edde7c42-11dd-4610-b9ed-54e3b56bbe0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b77ece7d36446c3a184e5a43a8939f6'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:39.007659 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1284/2551] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:19:38 2022] DELETE /identity/v3/users/5b77ece7d36446c3a184e5a43a8939f6 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:19:39.012690 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8dd8d-21ae-4428-af4b-5f0f9ac6d9a4 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:39.013122 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8dd8d-21ae-4428-af4b-5f0f9ac6d9a4 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:39.013467 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8dd8d-21ae-4428-af4b-5f0f9ac6d9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:39.035022 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-36e8dd8d-21ae-4428-af4b-5f0f9ac6d9a4 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:39.036350 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-36e8dd8d-21ae-4428-af4b-5f0f9ac6d9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:39.041811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139440c1-717a-4901-a939-24d45893da0a None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:39.104213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1285/2552] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:39.107736 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0160c53b-5f22-4f4b-bb14-34d8f0494b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:39.108590 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160c53b-5f22-4f4b-bb14-34d8f0494b81 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:39.108711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160c53b-5f22-4f4b-bb14-34d8f0494b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:39.108894 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160c53b-5f22-4f4b-bb14-34d8f0494b81 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:39.109523 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0160c53b-5f22-4f4b-bb14-34d8f0494b81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:39.111484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0160c53b-5f22-4f4b-bb14-34d8f0494b81 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:39.114437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8dd8d-21ae-4428-af4b-5f0f9ac6d9a4 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:39.115836 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-36e8dd8d-21ae-4428-af4b-5f0f9ac6d9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:39.178971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8dd8d-21ae-4428-af4b-5f0f9ac6d9a4 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:39.183336 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a38348-9559-468d-b8a9-ef55903f70e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:39.183939 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a38348-9559-468d-b8a9-ef55903f70e1 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:39.184254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a38348-9559-468d-b8a9-ef55903f70e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:39.184525 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a38348-9559-468d-b8a9-ef55903f70e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:39.193748 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a38348-9559-468d-b8a9-ef55903f70e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:39.195281 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a38348-9559-468d-b8a9-ef55903f70e1 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:39.204810 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1286/2553] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:19:39.270708 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1268/2554] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:19:38 2022] POST /identity/v3/projects => generated 373 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:19:39.271762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a38348-9559-468d-b8a9-ef55903f70e1 service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:39.281724 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a38348-9559-468d-b8a9-ef55903f70e1 service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:39.323288 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:39.339519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d980e09-796b-48cf-94c5-597275c806f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:39.340187 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d980e09-796b-48cf-94c5-597275c806f3 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:39.340542 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d980e09-796b-48cf-94c5-597275c806f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:39.340814 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d980e09-796b-48cf-94c5-597275c806f3 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:39.341435 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d980e09-796b-48cf-94c5-597275c806f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:39.343565 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d980e09-796b-48cf-94c5-597275c806f3 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:39.463521 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1287/2555] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:19:39 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:19:39.467470 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0160c53b-5f22-4f4b-bb14-34d8f0494b81 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:39.482641 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0160c53b-5f22-4f4b-bb14-34d8f0494b81 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:39.491262 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af807312-4468-4f7a-bf68-824066349f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:39.492558 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af807312-4468-4f7a-bf68-824066349f7a admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:39.493254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af807312-4468-4f7a-bf68-824066349f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:39.493768 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af807312-4468-4f7a-bf68-824066349f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:39.494431 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af807312-4468-4f7a-bf68-824066349f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:39.496458 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af807312-4468-4f7a-bf68-824066349f7a admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:39.512182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1269/2556] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:39.514526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d980e09-796b-48cf-94c5-597275c806f3 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:39.526688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d980e09-796b-48cf-94c5-597275c806f3 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:39.535260 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2cb26-d157-49ff-9718-307f12941f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:39.536349 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2cb26-d157-49ff-9718-307f12941f26 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:39.536731 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2cb26-d157-49ff-9718-307f12941f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:39.537102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2cb26-d157-49ff-9718-307f12941f26 admin admin] PATH_INFO: `/v3/projects/77f77aaa012b4618970937324297c220/users/24d56fdb8c3a47d7918566aeb379a313/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:39.555250 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2cb26-d157-49ff-9718-307f12941f26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77f77aaa012b4618970937324297c220, user_id=24d56fdb8c3a47d7918566aeb379a313, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:39.558026 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2cb26-d157-49ff-9718-307f12941f26 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:39.706131 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1288/2557] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:39 2022] PUT /identity/v3/projects/77f77aaa012b4618970937324297c220/users/24d56fdb8c3a47d7918566aeb379a313/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:39.707301 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af807312-4468-4f7a-bf68-824066349f7a admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:39.709328 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-af807312-4468-4f7a-bf68-824066349f7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:39.709493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2cb26-d157-49ff-9718-307f12941f26 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:39.725492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af807312-4468-4f7a-bf68-824066349f7a admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:39.873376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:39.886702 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d3288-8bb9-4070-b044-30477f1e5b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:39.888521 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d3288-8bb9-4070-b044-30477f1e5b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:39.888521 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d3288-8bb9-4070-b044-30477f1e5b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:39.888521 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d3288-8bb9-4070-b044-30477f1e5b9e admin admin] PATH_INFO: `/v3/projects/dc6f6ac628b54aaca55353f2fec3add8` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:39.889145 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d3288-8bb9-4070-b044-30477f1e5b9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc6f6ac628b54aaca55353f2fec3add8)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:39.891300 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d3288-8bb9-4070-b044-30477f1e5b9e admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:39.898411 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2cb26-d157-49ff-9718-307f12941f26 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:39.903633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70a54d-c52a-4126-a3b1-368eef7dc1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:39.904407 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70a54d-c52a-4126-a3b1-368eef7dc1da admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:39.904745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70a54d-c52a-4126-a3b1-368eef7dc1da admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:39.905076 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70a54d-c52a-4126-a3b1-368eef7dc1da admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:39.905721 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70a54d-c52a-4126-a3b1-368eef7dc1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:39.907711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70a54d-c52a-4126-a3b1-368eef7dc1da admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:39.921972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1289/2558] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:39.926750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70a54d-c52a-4126-a3b1-368eef7dc1da admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:39.937511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70a54d-c52a-4126-a3b1-368eef7dc1da admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:39.942325 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d770b-1c5a-47ba-9112-3b3346eec2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:39.942988 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d770b-1c5a-47ba-9112-3b3346eec2a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:39.943297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d770b-1c5a-47ba-9112-3b3346eec2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:39.943571 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d770b-1c5a-47ba-9112-3b3346eec2a7 admin admin] PATH_INFO: `/v3/projects/77f77aaa012b4618970937324297c220/users/24d56fdb8c3a47d7918566aeb379a313/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:39.946227 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d770b-1c5a-47ba-9112-3b3346eec2a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77f77aaa012b4618970937324297c220, user_id=24d56fdb8c3a47d7918566aeb379a313, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:39.948930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d770b-1c5a-47ba-9112-3b3346eec2a7 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:39.959206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1290/2559] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:39 2022] PUT /identity/v3/projects/77f77aaa012b4618970937324297c220/users/24d56fdb8c3a47d7918566aeb379a313/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:39.971368 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086c0ce-e8e8-4544-8054-3c3711d27e2b None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:39.972399 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086c0ce-e8e8-4544-8054-3c3711d27e2b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:39.972745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086c0ce-e8e8-4544-8054-3c3711d27e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:40.001777 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-9086c0ce-e8e8-4544-8054-3c3711d27e2b None None] MFA Rules not processed for user `24d56fdb8c3a47d7918566aeb379a313`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:40.004020 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-9086c0ce-e8e8-4544-8054-3c3711d27e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:40.030396 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-263d3288-8bb9-4070-b044-30477f1e5b9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc6f6ac628b54aaca55353f2fec3add8', 'request_id': 'req-263d3288-8bb9-4070-b044-30477f1e5b9e'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:40.087949 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1291/2560] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:19:40.099065 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086c0ce-e8e8-4544-8054-3c3711d27e2b None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:40.100430 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-9086c0ce-e8e8-4544-8054-3c3711d27e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:40.118796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:40.138983 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-263d3288-8bb9-4070-b044-30477f1e5b9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc6f6ac628b54aaca55353f2fec3add8', 'request_id': 'req-263d3288-8bb9-4070-b044-30477f1e5b9e'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:40.145251 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-263d3288-8bb9-4070-b044-30477f1e5b9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc6f6ac628b54aaca55353f2fec3add8', 'request_id': 'req-263d3288-8bb9-4070-b044-30477f1e5b9e'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:40.158158 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-263d3288-8bb9-4070-b044-30477f1e5b9e admin admin] The token cache is being invalidate because project dc6f6ac628b54aaca55353f2fec3add8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:40.158360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086c0ce-e8e8-4544-8054-3c3711d27e2b None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:40.158483 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-263d3288-8bb9-4070-b044-30477f1e5b9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:40.159020 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-263d3288-8bb9-4070-b044-30477f1e5b9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:40.160656 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1270/2561] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:39 2022] DELETE /identity/v3/projects/dc6f6ac628b54aaca55353f2fec3add8 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:40.164548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d3288-8bb9-4070-b044-30477f1e5b9e admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:40.166185 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-263d3288-8bb9-4070-b044-30477f1e5b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:40.213074 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec063cd-5020-44b2-9666-97c4426d0a7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:40.213687 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec063cd-5020-44b2-9666-97c4426d0a7c service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:40.213995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec063cd-5020-44b2-9666-97c4426d0a7c service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:40.214269 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec063cd-5020-44b2-9666-97c4426d0a7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:40.216072 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-eec063cd-5020-44b2-9666-97c4426d0a7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:40.237336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d3288-8bb9-4070-b044-30477f1e5b9e admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:40.242094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe7d22-da9c-4b0d-aeb1-7b22d5f4479d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:40.242730 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe7d22-da9c-4b0d-aeb1-7b22d5f4479d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:40.243013 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe7d22-da9c-4b0d-aeb1-7b22d5f4479d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:40.243302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe7d22-da9c-4b0d-aeb1-7b22d5f4479d admin admin] PATH_INFO: `/v3/users/1fb20b35287f4fe2bf5d25c9a5198cad` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:40.244470 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe7d22-da9c-4b0d-aeb1-7b22d5f4479d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fb20b35287f4fe2bf5d25c9a5198cad)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:40.246448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe7d22-da9c-4b0d-aeb1-7b22d5f4479d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:40.282281 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec063cd-5020-44b2-9666-97c4426d0a7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:40.283717 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec063cd-5020-44b2-9666-97c4426d0a7c service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:40.293581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1292/2562] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:19:40.513570 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1cfe7d22-da9c-4b0d-aeb1-7b22d5f4479d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fb20b35287f4fe2bf5d25c9a5198cad'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:40.520485 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1cfe7d22-da9c-4b0d-aeb1-7b22d5f4479d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fb20b35287f4fe2bf5d25c9a5198cad'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:40.521156 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1cfe7d22-da9c-4b0d-aeb1-7b22d5f4479d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fb20b35287f4fe2bf5d25c9a5198cad'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:40.533389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1cfe7d22-da9c-4b0d-aeb1-7b22d5f4479d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fb20b35287f4fe2bf5d25c9a5198cad'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:40.534024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1cfe7d22-da9c-4b0d-aeb1-7b22d5f4479d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fb20b35287f4fe2bf5d25c9a5198cad'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:40.594554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec063cd-5020-44b2-9666-97c4426d0a7c service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:40.596533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-eec063cd-5020-44b2-9666-97c4426d0a7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:40.608177 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec063cd-5020-44b2-9666-97c4426d0a7c service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:40.612084 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ce716-328e-4ac6-83fe-6b326468a95d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:40.612784 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ce716-328e-4ac6-83fe-6b326468a95d service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:40.613125 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ce716-328e-4ac6-83fe-6b326468a95d service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:40.613459 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ce716-328e-4ac6-83fe-6b326468a95d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:40.615339 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-fc2ce716-328e-4ac6-83fe-6b326468a95d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:40.630105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:40.645567 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1271/2563] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:19:40 2022] DELETE /identity/v3/users/1fb20b35287f4fe2bf5d25c9a5198cad => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:19:40.689714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ce716-328e-4ac6-83fe-6b326468a95d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:40.691088 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ce716-328e-4ac6-83fe-6b326468a95d service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:40.701694 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1293/2564] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:19:41.159538 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe7d22-da9c-4b0d-aeb1-7b22d5f4479d admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:41.161019 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-1cfe7d22-da9c-4b0d-aeb1-7b22d5f4479d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:41.242440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe7d22-da9c-4b0d-aeb1-7b22d5f4479d admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:41.247060 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b6a48-f827-48d3-b93c-7a81a501e42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:41.247780 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b6a48-f827-48d3-b93c-7a81a501e42c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:41.248059 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b6a48-f827-48d3-b93c-7a81a501e42c admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:41.248355 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b6a48-f827-48d3-b93c-7a81a501e42c admin admin] PATH_INFO: `/v3/projects/6c79c4711ac04382b542d5b78c298ed5` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:41.249499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b6a48-f827-48d3-b93c-7a81a501e42c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c79c4711ac04382b542d5b78c298ed5)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:41.251367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b6a48-f827-48d3-b93c-7a81a501e42c admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:41.388019 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b55b6a48-f827-48d3-b93c-7a81a501e42c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c79c4711ac04382b542d5b78c298ed5', 'request_id': 'req-b55b6a48-f827-48d3-b93c-7a81a501e42c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:41.494880 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b55b6a48-f827-48d3-b93c-7a81a501e42c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c79c4711ac04382b542d5b78c298ed5', 'request_id': 'req-b55b6a48-f827-48d3-b93c-7a81a501e42c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:41.501367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b55b6a48-f827-48d3-b93c-7a81a501e42c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c79c4711ac04382b542d5b78c298ed5', 'request_id': 'req-b55b6a48-f827-48d3-b93c-7a81a501e42c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:41.513479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b55b6a48-f827-48d3-b93c-7a81a501e42c admin admin] The token cache is being invalidate because project 6c79c4711ac04382b542d5b78c298ed5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:41.513775 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b55b6a48-f827-48d3-b93c-7a81a501e42c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:41.514412 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-b55b6a48-f827-48d3-b93c-7a81a501e42c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:41.515938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1272/2565] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:41 2022] DELETE /identity/v3/projects/6c79c4711ac04382b542d5b78c298ed5 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:41.519864 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ce716-328e-4ac6-83fe-6b326468a95d service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:41.531890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ce716-328e-4ac6-83fe-6b326468a95d service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:41.548558 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:41.589773 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9b170-3f08-4b72-b13c-f9778758e950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:41.590876 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9b170-3f08-4b72-b13c-f9778758e950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:41.591165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9b170-3f08-4b72-b13c-f9778758e950 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:41.591444 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9b170-3f08-4b72-b13c-f9778758e950 admin admin] PATH_INFO: `/v3/users/cd03ed4e6eb24bea85b865a7debec2d9` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:41.592740 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9b170-3f08-4b72-b13c-f9778758e950 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd03ed4e6eb24bea85b865a7debec2d9)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:41.594744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9b170-3f08-4b72-b13c-f9778758e950 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:41.816939 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-53e9b170-3f08-4b72-b13c-f9778758e950 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd03ed4e6eb24bea85b865a7debec2d9'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:41.817579 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-53e9b170-3f08-4b72-b13c-f9778758e950 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd03ed4e6eb24bea85b865a7debec2d9'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:41.826817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-53e9b170-3f08-4b72-b13c-f9778758e950 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd03ed4e6eb24bea85b865a7debec2d9'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:41.935041 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-53e9b170-3f08-4b72-b13c-f9778758e950 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd03ed4e6eb24bea85b865a7debec2d9'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:41.935661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-53e9b170-3f08-4b72-b13c-f9778758e950 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd03ed4e6eb24bea85b865a7debec2d9'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:41.954247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1294/2566] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:19:41 2022] DELETE /identity/v3/users/cd03ed4e6eb24bea85b865a7debec2d9 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:42.176774 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b6a48-f827-48d3-b93c-7a81a501e42c admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:42.178331 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b55b6a48-f827-48d3-b93c-7a81a501e42c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:42.238281 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b6a48-f827-48d3-b93c-7a81a501e42c admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:42.291767 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:42.296835 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe17c4-97f9-403a-a293-f72a9e973013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:42.297455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe17c4-97f9-403a-a293-f72a9e973013 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:42.297730 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe17c4-97f9-403a-a293-f72a9e973013 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:42.297994 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe17c4-97f9-403a-a293-f72a9e973013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:42.300062 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-82fe17c4-97f9-403a-a293-f72a9e973013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:42.367926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe17c4-97f9-403a-a293-f72a9e973013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:42.368322 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe17c4-97f9-403a-a293-f72a9e973013 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:42.378201 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1273/2567] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:19:42.402215 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9b170-3f08-4b72-b13c-f9778758e950 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:42.405041 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-53e9b170-3f08-4b72-b13c-f9778758e950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:42.422678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9b170-3f08-4b72-b13c-f9778758e950 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:42.429109 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a0f2f-d77d-400b-9d06-08eab51707bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:42.430674 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a0f2f-d77d-400b-9d06-08eab51707bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:42.430944 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a0f2f-d77d-400b-9d06-08eab51707bc admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:42.431224 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a0f2f-d77d-400b-9d06-08eab51707bc admin admin] PATH_INFO: `/v3/projects/f6e50055e84140c894898b1cc976dcff` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:42.432483 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a0f2f-d77d-400b-9d06-08eab51707bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6e50055e84140c894898b1cc976dcff)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:42.435313 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a0f2f-d77d-400b-9d06-08eab51707bc admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:42.568481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6a5a0f2f-d77d-400b-9d06-08eab51707bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6e50055e84140c894898b1cc976dcff', 'request_id': 'req-6a5a0f2f-d77d-400b-9d06-08eab51707bc'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:42.575645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6a5a0f2f-d77d-400b-9d06-08eab51707bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6e50055e84140c894898b1cc976dcff', 'request_id': 'req-6a5a0f2f-d77d-400b-9d06-08eab51707bc'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:42.581556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6a5a0f2f-d77d-400b-9d06-08eab51707bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6e50055e84140c894898b1cc976dcff', 'request_id': 'req-6a5a0f2f-d77d-400b-9d06-08eab51707bc'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:42.670462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6a5a0f2f-d77d-400b-9d06-08eab51707bc admin admin] The token cache is being invalidate because project f6e50055e84140c894898b1cc976dcff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:42.670750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6a5a0f2f-d77d-400b-9d06-08eab51707bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:42.671413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6a5a0f2f-d77d-400b-9d06-08eab51707bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:42.673224 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1295/2568] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:42 2022] DELETE /identity/v3/projects/f6e50055e84140c894898b1cc976dcff => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:42.679736 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98e8bd-b0a1-421d-ac85-5f821e474fc8 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:42.680555 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98e8bd-b0a1-421d-ac85-5f821e474fc8 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:42.680555 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98e8bd-b0a1-421d-ac85-5f821e474fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:42.703128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-3f98e8bd-b0a1-421d-ac85-5f821e474fc8 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:42.704406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3f98e8bd-b0a1-421d-ac85-5f821e474fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:42.792435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1274/2569] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:42.801304 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0e976-d688-419b-95e7-47e687c222a9 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:42.801673 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0e976-d688-419b-95e7-47e687c222a9 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:42.801943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0e976-d688-419b-95e7-47e687c222a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:42.823644 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-a1a0e976-d688-419b-95e7-47e687c222a9 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:42.824881 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a1a0e976-d688-419b-95e7-47e687c222a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:42.833032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1296/2570] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:19:42.837750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98e8bd-b0a1-421d-ac85-5f821e474fc8 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:42.847705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98e8bd-b0a1-421d-ac85-5f821e474fc8 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:42.852494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138cebe3-779d-4d9a-96cf-b22e9c4a1509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:42.857119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cebe3-779d-4d9a-96cf-b22e9c4a1509 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:42.857119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cebe3-779d-4d9a-96cf-b22e9c4a1509 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:42.857119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138cebe3-779d-4d9a-96cf-b22e9c4a1509 admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:42.857119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138cebe3-779d-4d9a-96cf-b22e9c4a1509 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:19:42.857119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138cebe3-779d-4d9a-96cf-b22e9c4a1509 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:42.857119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138cebe3-779d-4d9a-96cf-b22e9c4a1509 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:42.869378 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1275/2571] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:19:42.873862 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0e976-d688-419b-95e7-47e687c222a9 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:42.884010 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0e976-d688-419b-95e7-47e687c222a9 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:42.888913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a12b9-b888-49bf-90df-17c201fa2349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:42.889462 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a12b9-b888-49bf-90df-17c201fa2349 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:42.889743 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a12b9-b888-49bf-90df-17c201fa2349 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:42.890014 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a12b9-b888-49bf-90df-17c201fa2349 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:42.890655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974a12b9-b888-49bf-90df-17c201fa2349 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:42.892630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974a12b9-b888-49bf-90df-17c201fa2349 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:42.908665 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:43.012055 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1297/2572] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:19:42 2022] POST /identity/v3/projects => generated 381 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:19:43.016281 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138cebe3-779d-4d9a-96cf-b22e9c4a1509 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:43.027210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138cebe3-779d-4d9a-96cf-b22e9c4a1509 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:43.094005 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc709b04-b744-4a53-9be2-2d916c735af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:43.094632 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc709b04-b744-4a53-9be2-2d916c735af4 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:43.094906 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc709b04-b744-4a53-9be2-2d916c735af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:43.095175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc709b04-b744-4a53-9be2-2d916c735af4 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:43.095808 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc709b04-b744-4a53-9be2-2d916c735af4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:43.097803 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc709b04-b744-4a53-9be2-2d916c735af4 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:43.129247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a12b9-b888-49bf-90df-17c201fa2349 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:43.131725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-974a12b9-b888-49bf-90df-17c201fa2349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:43.146669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a12b9-b888-49bf-90df-17c201fa2349 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:43.153402 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992e321-499c-44a2-b896-00b91d260e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:43.154085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992e321-499c-44a2-b896-00b91d260e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:43.154365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992e321-499c-44a2-b896-00b91d260e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:43.154894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992e321-499c-44a2-b896-00b91d260e75 admin admin] PATH_INFO: `/v3/users/484c09f238bd4b0d91b6aeaee913f7d5` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:43.156714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992e321-499c-44a2-b896-00b91d260e75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=484c09f238bd4b0d91b6aeaee913f7d5)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:43.158809 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992e321-499c-44a2-b896-00b91d260e75 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:43.196313 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1276/2573] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:19:43 2022] POST /identity/v3/users => generated 291 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:19:43.200236 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc709b04-b744-4a53-9be2-2d916c735af4 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:43.214159 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc709b04-b744-4a53-9be2-2d916c735af4 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:43.219069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d55cc8-f6d0-4b1d-af76-d54346d53fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:43.219713 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d55cc8-f6d0-4b1d-af76-d54346d53fdd admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:43.220017 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d55cc8-f6d0-4b1d-af76-d54346d53fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:43.220341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d55cc8-f6d0-4b1d-af76-d54346d53fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:43.221014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d55cc8-f6d0-4b1d-af76-d54346d53fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:43.222630 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d55cc8-f6d0-4b1d-af76-d54346d53fdd admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:43.228260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:43.233743 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1277/2574] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:43.237579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d55cc8-f6d0-4b1d-af76-d54346d53fdd admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:43.247175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d55cc8-f6d0-4b1d-af76-d54346d53fdd admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:43.251880 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451347d4-13a9-4e0a-8057-42f7d6c186ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:43.252484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451347d4-13a9-4e0a-8057-42f7d6c186ea admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:43.252757 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451347d4-13a9-4e0a-8057-42f7d6c186ea admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:43.253022 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451347d4-13a9-4e0a-8057-42f7d6c186ea admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:43.253638 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451347d4-13a9-4e0a-8057-42f7d6c186ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:43.255164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451347d4-13a9-4e0a-8057-42f7d6c186ea admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:43.265626 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1278/2575] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:19:43.269480 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451347d4-13a9-4e0a-8057-42f7d6c186ea admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:43.280131 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451347d4-13a9-4e0a-8057-42f7d6c186ea admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:43.284896 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c54ed3-b05c-4745-9934-82f33c0d1d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:43.285541 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c54ed3-b05c-4745-9934-82f33c0d1d2c admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:43.285815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c54ed3-b05c-4745-9934-82f33c0d1d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:43.286086 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c54ed3-b05c-4745-9934-82f33c0d1d2c admin admin] PATH_INFO: `/v3/projects/a4e57fef7e1444568ed9393f3d604632/users/e9dfa3ceaae24647b775313959070ff3/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:43.302074 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c54ed3-b05c-4745-9934-82f33c0d1d2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e57fef7e1444568ed9393f3d604632, user_id=e9dfa3ceaae24647b775313959070ff3, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:43.305710 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c54ed3-b05c-4745-9934-82f33c0d1d2c admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:43.383817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1992e321-499c-44a2-b896-00b91d260e75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '484c09f238bd4b0d91b6aeaee913f7d5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:43.384488 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1992e321-499c-44a2-b896-00b91d260e75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '484c09f238bd4b0d91b6aeaee913f7d5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:43.391758 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1992e321-499c-44a2-b896-00b91d260e75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '484c09f238bd4b0d91b6aeaee913f7d5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:43.409960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1279/2576] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:43 2022] PUT /identity/v3/projects/a4e57fef7e1444568ed9393f3d604632/users/e9dfa3ceaae24647b775313959070ff3/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:19:43.421572 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcebb4-f0c7-4f9f-a4f6-37a529a0c62c None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:43.421942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcebb4-f0c7-4f9f-a4f6-37a529a0c62c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:43.422117 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcebb4-f0c7-4f9f-a4f6-37a529a0c62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:43.450835 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-6dbcebb4-f0c7-4f9f-a4f6-37a529a0c62c None None] MFA Rules not processed for user `e9dfa3ceaae24647b775313959070ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:43.452075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6dbcebb4-f0c7-4f9f-a4f6-37a529a0c62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:43.516107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1280/2577] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:19:43.520774 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbcebb4-f0c7-4f9f-a4f6-37a529a0c62c None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:43.522371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6dbcebb4-f0c7-4f9f-a4f6-37a529a0c62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:43.523865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1992e321-499c-44a2-b896-00b91d260e75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '484c09f238bd4b0d91b6aeaee913f7d5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:43.524524 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1992e321-499c-44a2-b896-00b91d260e75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '484c09f238bd4b0d91b6aeaee913f7d5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:43.543160 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1298/2578] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:19:43 2022] DELETE /identity/v3/users/484c09f238bd4b0d91b6aeaee913f7d5 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:43.590300 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbcebb4-f0c7-4f9f-a4f6-37a529a0c62c None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:43.650021 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e36509-590b-4b2d-b5af-615fe4127a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:43.650646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e36509-590b-4b2d-b5af-615fe4127a3e admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:43.650929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e36509-590b-4b2d-b5af-615fe4127a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:43.651258 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e36509-590b-4b2d-b5af-615fe4127a3e admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:43.651891 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e36509-590b-4b2d-b5af-615fe4127a3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:43.653814 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e36509-590b-4b2d-b5af-615fe4127a3e admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:43.809835 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1281/2579] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:19:43 2022] POST /identity/v3/projects => generated 383 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:19:43.813592 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992e321-499c-44a2-b896-00b91d260e75 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:43.823771 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992e321-499c-44a2-b896-00b91d260e75 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:43.881507 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c905e-7a82-4471-841c-53d34c396259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:43.882155 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c905e-7a82-4471-841c-53d34c396259 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:43.882639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c905e-7a82-4471-841c-53d34c396259 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:43.882984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c905e-7a82-4471-841c-53d34c396259 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:43.884328 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c905e-7a82-4471-841c-53d34c396259 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:43.895921 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e36509-590b-4b2d-b5af-615fe4127a3e admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:43.896816 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c905e-7a82-4471-841c-53d34c396259 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:43.897104 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f2e36509-590b-4b2d-b5af-615fe4127a3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:43.902653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e36509-590b-4b2d-b5af-615fe4127a3e admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:43.907656 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bda115-905b-40fb-9749-ee528b7eae8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:43.908355 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bda115-905b-40fb-9749-ee528b7eae8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:43.908631 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bda115-905b-40fb-9749-ee528b7eae8f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:43.908897 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bda115-905b-40fb-9749-ee528b7eae8f admin admin] PATH_INFO: `/v3/projects/2c0dcefe769247eaa49da038e336bc8a` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:43.910055 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bda115-905b-40fb-9749-ee528b7eae8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c0dcefe769247eaa49da038e336bc8a)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:43.912005 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bda115-905b-40fb-9749-ee528b7eae8f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:43.986498 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1299/2580] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:19:43 2022] POST /identity/v3/users => generated 298 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:19:43.990584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c905e-7a82-4471-841c-53d34c396259 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:44.001918 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c905e-7a82-4471-841c-53d34c396259 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:44.006844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a17b26-90a1-4e7a-b06c-45de0fddb24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:44.007505 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a17b26-90a1-4e7a-b06c-45de0fddb24b admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:44.007997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a17b26-90a1-4e7a-b06c-45de0fddb24b admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:44.008361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a17b26-90a1-4e7a-b06c-45de0fddb24b admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:44.009012 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a17b26-90a1-4e7a-b06c-45de0fddb24b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:44.010612 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a17b26-90a1-4e7a-b06c-45de0fddb24b admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:44.023442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1300/2581] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:19:44.031261 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a17b26-90a1-4e7a-b06c-45de0fddb24b admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:44.033215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c0bda115-905b-40fb-9749-ee528b7eae8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c0dcefe769247eaa49da038e336bc8a', 'request_id': 'req-c0bda115-905b-40fb-9749-ee528b7eae8f'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:44.042824 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a17b26-90a1-4e7a-b06c-45de0fddb24b admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:44.099835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc82d8-4a17-4b3a-87cf-b3a9a1518405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:44.100523 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc82d8-4a17-4b3a-87cf-b3a9a1518405 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:44.100803 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc82d8-4a17-4b3a-87cf-b3a9a1518405 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:44.101783 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc82d8-4a17-4b3a-87cf-b3a9a1518405 admin admin] PATH_INFO: `/v3/projects/42b920b60bb5477f9c2f8a0fb6c1d679/users/26a6e282d8fe4060a634784acf1eaefd/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:44.111595 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c0bda115-905b-40fb-9749-ee528b7eae8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c0dcefe769247eaa49da038e336bc8a', 'request_id': 'req-c0bda115-905b-40fb-9749-ee528b7eae8f'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:44.117024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc82d8-4a17-4b3a-87cf-b3a9a1518405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b920b60bb5477f9c2f8a0fb6c1d679, user_id=26a6e282d8fe4060a634784acf1eaefd, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:44.117892 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c0bda115-905b-40fb-9749-ee528b7eae8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c0dcefe769247eaa49da038e336bc8a', 'request_id': 'req-c0bda115-905b-40fb-9749-ee528b7eae8f'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:44.119679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc82d8-4a17-4b3a-87cf-b3a9a1518405 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:44.130091 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c0bda115-905b-40fb-9749-ee528b7eae8f admin admin] The token cache is being invalidate because project 2c0dcefe769247eaa49da038e336bc8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:44.130378 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c0bda115-905b-40fb-9749-ee528b7eae8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:44.131110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c0bda115-905b-40fb-9749-ee528b7eae8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:44.132830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1282/2582] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:43 2022] DELETE /identity/v3/projects/2c0dcefe769247eaa49da038e336bc8a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:44.213856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1301/2583] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:44 2022] PUT /identity/v3/projects/42b920b60bb5477f9c2f8a0fb6c1d679/users/26a6e282d8fe4060a634784acf1eaefd/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:19:44.217615 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bda115-905b-40fb-9749-ee528b7eae8f admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:44.219233 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-c0bda115-905b-40fb-9749-ee528b7eae8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:44.261383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bda115-905b-40fb-9749-ee528b7eae8f admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:44.274129 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:INSERT=5,keystone:SELECT=85,keystone:DELETE=10 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:44.324440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:44.348222 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b2b71-4fa0-4ed0-b3e3-e1b51554f7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:44.348830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b2b71-4fa0-4ed0-b3e3-e1b51554f7f8 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:44.349104 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b2b71-4fa0-4ed0-b3e3-e1b51554f7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:44.349370 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b2b71-4fa0-4ed0-b3e3-e1b51554f7f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:44.349996 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b2b71-4fa0-4ed0-b3e3-e1b51554f7f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:44.352165 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b2b71-4fa0-4ed0-b3e3-e1b51554f7f8 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:44.362069 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1283/2584] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:44.366319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc82d8-4a17-4b3a-87cf-b3a9a1518405 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:44.377021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc82d8-4a17-4b3a-87cf-b3a9a1518405 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:44.381881 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114d52c-8b3a-4052-b0ab-aab013376f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:44.382606 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114d52c-8b3a-4052-b0ab-aab013376f00 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:44.382967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114d52c-8b3a-4052-b0ab-aab013376f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:44.383239 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114d52c-8b3a-4052-b0ab-aab013376f00 admin admin] PATH_INFO: `/v3/projects/42b920b60bb5477f9c2f8a0fb6c1d679/users/26a6e282d8fe4060a634784acf1eaefd/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:44.385295 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114d52c-8b3a-4052-b0ab-aab013376f00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b920b60bb5477f9c2f8a0fb6c1d679, user_id=26a6e282d8fe4060a634784acf1eaefd, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:44.387976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114d52c-8b3a-4052-b0ab-aab013376f00 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:44.399585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1302/2585] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:44 2022] PUT /identity/v3/projects/42b920b60bb5477f9c2f8a0fb6c1d679/users/26a6e282d8fe4060a634784acf1eaefd/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:44.470403 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d6a6a-4305-4cc3-8588-eb2a2d093106 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:44.470670 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d6a6a-4305-4cc3-8588-eb2a2d093106 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:44.470915 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d6a6a-4305-4cc3-8588-eb2a2d093106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:44.499367 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-3c5d6a6a-4305-4cc3-8588-eb2a2d093106 None None] MFA Rules not processed for user `26a6e282d8fe4060a634784acf1eaefd`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:44.500835 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3c5d6a6a-4305-4cc3-8588-eb2a2d093106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:44.572379 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1284/2586] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:44 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:44.584105 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114d52c-8b3a-4052-b0ab-aab013376f00 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:44.586922 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c114d52c-8b3a-4052-b0ab-aab013376f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:44.644664 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114d52c-8b3a-4052-b0ab-aab013376f00 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:44.650257 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc56d7-add8-4f9e-baef-b53f9f22d2fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:44.650866 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc56d7-add8-4f9e-baef-b53f9f22d2fa service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:44.651196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc56d7-add8-4f9e-baef-b53f9f22d2fa service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:44.651472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc56d7-add8-4f9e-baef-b53f9f22d2fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:44.662826 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc56d7-add8-4f9e-baef-b53f9f22d2fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:44.664422 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc56d7-add8-4f9e-baef-b53f9f22d2fa service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:44.673977 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1303/2587] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:19:44.739290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:44.858579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d6a6a-4305-4cc3-8588-eb2a2d093106 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:44.870693 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d6a6a-4305-4cc3-8588-eb2a2d093106 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:44.877647 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0f84c-9d1f-4cce-83ed-0dd9d0b38aa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:44.878267 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0f84c-9d1f-4cce-83ed-0dd9d0b38aa7 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:44.878538 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0f84c-9d1f-4cce-83ed-0dd9d0b38aa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:44.878802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0f84c-9d1f-4cce-83ed-0dd9d0b38aa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:44.880544 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-5cc0f84c-9d1f-4cce-83ed-0dd9d0b38aa7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:44.942961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0f84c-9d1f-4cce-83ed-0dd9d0b38aa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:44.944344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0f84c-9d1f-4cce-83ed-0dd9d0b38aa7 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:44.954587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1285/2588] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:19:45.549774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc56d7-add8-4f9e-baef-b53f9f22d2fa service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:45.618391 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc56d7-add8-4f9e-baef-b53f9f22d2fa service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:45.622871 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11206480-6b2e-4e0d-b8fe-d91201a9a762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:45.623472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11206480-6b2e-4e0d-b8fe-d91201a9a762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:45.623746 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11206480-6b2e-4e0d-b8fe-d91201a9a762 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:45.624011 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11206480-6b2e-4e0d-b8fe-d91201a9a762 admin admin] PATH_INFO: `/v3/users/e9dfa3ceaae24647b775313959070ff3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:45.625169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11206480-6b2e-4e0d-b8fe-d91201a9a762 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9dfa3ceaae24647b775313959070ff3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:45.627063 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11206480-6b2e-4e0d-b8fe-d91201a9a762 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:45.857229 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-11206480-6b2e-4e0d-b8fe-d91201a9a762 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9dfa3ceaae24647b775313959070ff3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:45.857899 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-11206480-6b2e-4e0d-b8fe-d91201a9a762 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9dfa3ceaae24647b775313959070ff3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:45.864805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-11206480-6b2e-4e0d-b8fe-d91201a9a762 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9dfa3ceaae24647b775313959070ff3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:45.868188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:45.941057 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-11206480-6b2e-4e0d-b8fe-d91201a9a762 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9dfa3ceaae24647b775313959070ff3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:45.941761 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-11206480-6b2e-4e0d-b8fe-d91201a9a762 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9dfa3ceaae24647b775313959070ff3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:45.971164 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1304/2589] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:19:45 2022] DELETE /identity/v3/users/e9dfa3ceaae24647b775313959070ff3 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:45.979889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b04243-4642-40ef-aed7-3f1a8c846c5b None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:45.980389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b04243-4642-40ef-aed7-3f1a8c846c5b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:45.980509 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b04243-4642-40ef-aed7-3f1a8c846c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:46.000603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-f6b04243-4642-40ef-aed7-3f1a8c846c5b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:46.001807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f6b04243-4642-40ef-aed7-3f1a8c846c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:46.068531 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1286/2590] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:46.075395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11206480-6b2e-4e0d-b8fe-d91201a9a762 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:46.076828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-11206480-6b2e-4e0d-b8fe-d91201a9a762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:46.135623 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11206480-6b2e-4e0d-b8fe-d91201a9a762 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:46.139238 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef289d57-b7c9-4a81-ab2d-c5ec6affd51d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:46.139890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef289d57-b7c9-4a81-ab2d-c5ec6affd51d service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:46.140185 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef289d57-b7c9-4a81-ab2d-c5ec6affd51d service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:46.140455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef289d57-b7c9-4a81-ab2d-c5ec6affd51d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:46.149834 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef289d57-b7c9-4a81-ab2d-c5ec6affd51d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:46.151251 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef289d57-b7c9-4a81-ab2d-c5ec6affd51d service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:46.160883 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1305/2591] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:19:46.588502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b04243-4642-40ef-aed7-3f1a8c846c5b None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:46.589943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f6b04243-4642-40ef-aed7-3f1a8c846c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:46.604361 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b04243-4642-40ef-aed7-3f1a8c846c5b None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:46.610254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719d61a-2c93-4a6b-925c-b0c406c2deb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:46.610952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719d61a-2c93-4a6b-925c-b0c406c2deb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:46.611250 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719d61a-2c93-4a6b-925c-b0c406c2deb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:46.611533 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719d61a-2c93-4a6b-925c-b0c406c2deb9 admin admin] PATH_INFO: `/v3/projects/a4e57fef7e1444568ed9393f3d604632` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:46.612961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719d61a-2c93-4a6b-925c-b0c406c2deb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4e57fef7e1444568ed9393f3d604632)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:46.614926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719d61a-2c93-4a6b-925c-b0c406c2deb9 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:46.788114 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-4719d61a-2c93-4a6b-925c-b0c406c2deb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4e57fef7e1444568ed9393f3d604632', 'request_id': 'req-4719d61a-2c93-4a6b-925c-b0c406c2deb9'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:46.866055 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-4719d61a-2c93-4a6b-925c-b0c406c2deb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4e57fef7e1444568ed9393f3d604632', 'request_id': 'req-4719d61a-2c93-4a6b-925c-b0c406c2deb9'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:46.873324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-4719d61a-2c93-4a6b-925c-b0c406c2deb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4e57fef7e1444568ed9393f3d604632', 'request_id': 'req-4719d61a-2c93-4a6b-925c-b0c406c2deb9'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:46.885646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-4719d61a-2c93-4a6b-925c-b0c406c2deb9 admin admin] The token cache is being invalidate because project a4e57fef7e1444568ed9393f3d604632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:46.885985 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-4719d61a-2c93-4a6b-925c-b0c406c2deb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:46.886675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-4719d61a-2c93-4a6b-925c-b0c406c2deb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:46.888340 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1287/2592] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:46 2022] DELETE /identity/v3/projects/a4e57fef7e1444568ed9393f3d604632 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:46.892369 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef289d57-b7c9-4a81-ab2d-c5ec6affd51d service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:46.902891 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef289d57-b7c9-4a81-ab2d-c5ec6affd51d service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:46.962732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c044f70-765e-4aa2-98f9-899a288bafeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:46.963370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c044f70-765e-4aa2-98f9-899a288bafeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:46.963645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c044f70-765e-4aa2-98f9-899a288bafeb admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:46.963912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c044f70-765e-4aa2-98f9-899a288bafeb admin admin] PATH_INFO: `/v3/users/26a6e282d8fe4060a634784acf1eaefd` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:46.965064 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c044f70-765e-4aa2-98f9-899a288bafeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=26a6e282d8fe4060a634784acf1eaefd)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:46.966974 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c044f70-765e-4aa2-98f9-899a288bafeb admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:47.177205 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1c044f70-765e-4aa2-98f9-899a288bafeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26a6e282d8fe4060a634784acf1eaefd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:47.177862 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1c044f70-765e-4aa2-98f9-899a288bafeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26a6e282d8fe4060a634784acf1eaefd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:47.185285 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1c044f70-765e-4aa2-98f9-899a288bafeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26a6e282d8fe4060a634784acf1eaefd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:47.255102 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1c044f70-765e-4aa2-98f9-899a288bafeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26a6e282d8fe4060a634784acf1eaefd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:47.255755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1c044f70-765e-4aa2-98f9-899a288bafeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26a6e282d8fe4060a634784acf1eaefd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:47.270168 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:47.276457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1306/2593] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:19:46 2022] DELETE /identity/v3/users/26a6e282d8fe4060a634784acf1eaefd => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:47.724805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719d61a-2c93-4a6b-925c-b0c406c2deb9 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:47.726522 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-4719d61a-2c93-4a6b-925c-b0c406c2deb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:47.739614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719d61a-2c93-4a6b-925c-b0c406c2deb9 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:47.751254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:47.802349 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c450aa2-6d52-471c-90b9-3af196586d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:47.803015 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c450aa2-6d52-471c-90b9-3af196586d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:47.803298 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c450aa2-6d52-471c-90b9-3af196586d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:47.803564 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c450aa2-6d52-471c-90b9-3af196586d0c admin admin] PATH_INFO: `/v3/projects/42b920b60bb5477f9c2f8a0fb6c1d679` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:47.806749 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c450aa2-6d52-471c-90b9-3af196586d0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=42b920b60bb5477f9c2f8a0fb6c1d679)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:47.808724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c450aa2-6d52-471c-90b9-3af196586d0c admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:47.941228 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2c450aa2-6d52-471c-90b9-3af196586d0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42b920b60bb5477f9c2f8a0fb6c1d679', 'request_id': 'req-2c450aa2-6d52-471c-90b9-3af196586d0c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:48.070713 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2c450aa2-6d52-471c-90b9-3af196586d0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42b920b60bb5477f9c2f8a0fb6c1d679', 'request_id': 'req-2c450aa2-6d52-471c-90b9-3af196586d0c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:48.077236 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2c450aa2-6d52-471c-90b9-3af196586d0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42b920b60bb5477f9c2f8a0fb6c1d679', 'request_id': 'req-2c450aa2-6d52-471c-90b9-3af196586d0c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:48.091293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2c450aa2-6d52-471c-90b9-3af196586d0c admin admin] The token cache is being invalidate because project 42b920b60bb5477f9c2f8a0fb6c1d679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:48.091293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2c450aa2-6d52-471c-90b9-3af196586d0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:48.091927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2c450aa2-6d52-471c-90b9-3af196586d0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:48.093545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1288/2594] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:47 2022] DELETE /identity/v3/projects/42b920b60bb5477f9c2f8a0fb6c1d679 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:48.099201 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218940e-863f-4709-a51e-16fed9bae866 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:48.099708 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218940e-863f-4709-a51e-16fed9bae866 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:48.099833 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218940e-863f-4709-a51e-16fed9bae866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:48.121584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-e218940e-863f-4709-a51e-16fed9bae866 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:48.122929 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e218940e-863f-4709-a51e-16fed9bae866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:48.184254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1307/2595] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:48.191763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad550f-1151-43b5-970f-5f63e5483007 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:48.192042 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad550f-1151-43b5-970f-5f63e5483007 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:48.192341 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad550f-1151-43b5-970f-5f63e5483007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:48.211890 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-dcad550f-1151-43b5-970f-5f63e5483007 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:48.213123 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-dcad550f-1151-43b5-970f-5f63e5483007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:48.221000 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1289/2596] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:48.225544 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218940e-863f-4709-a51e-16fed9bae866 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:48.235967 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218940e-863f-4709-a51e-16fed9bae866 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:48.240620 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86fae20-e591-4457-8c6c-0f37a8942268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:48.241206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fae20-e591-4457-8c6c-0f37a8942268 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:48.242163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fae20-e591-4457-8c6c-0f37a8942268 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:48.242424 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fae20-e591-4457-8c6c-0f37a8942268 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:48.243171 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86fae20-e591-4457-8c6c-0f37a8942268 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:19:48.243360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86fae20-e591-4457-8c6c-0f37a8942268 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:48.244977 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86fae20-e591-4457-8c6c-0f37a8942268 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:48.256921 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1308/2597] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:19:48.261110 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad550f-1151-43b5-970f-5f63e5483007 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:48.270445 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad550f-1151-43b5-970f-5f63e5483007 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:48.275380 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac6670-e30d-4995-8723-25c4aff4713a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:48.275981 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac6670-e30d-4995-8723-25c4aff4713a admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:48.276296 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac6670-e30d-4995-8723-25c4aff4713a admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:48.276563 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac6670-e30d-4995-8723-25c4aff4713a admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:48.277228 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac6670-e30d-4995-8723-25c4aff4713a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:48.279107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac6670-e30d-4995-8723-25c4aff4713a admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:48.364642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1290/2598] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:19:48 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:19:48.369615 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86fae20-e591-4457-8c6c-0f37a8942268 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:48.379987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86fae20-e591-4457-8c6c-0f37a8942268 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:48.438432 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4394d55-08fd-47af-9787-1db593bed852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:48.439040 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4394d55-08fd-47af-9787-1db593bed852 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:48.439319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4394d55-08fd-47af-9787-1db593bed852 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:48.439619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4394d55-08fd-47af-9787-1db593bed852 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:48.440289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4394d55-08fd-47af-9787-1db593bed852 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:48.442233 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4394d55-08fd-47af-9787-1db593bed852 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:48.541452 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1309/2599] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:19:48 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:19:48.545520 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac6670-e30d-4995-8723-25c4aff4713a admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:48.556436 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac6670-e30d-4995-8723-25c4aff4713a admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:48.561250 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd278a1-ddf7-4794-ae4a-04ed81ca7915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:48.561849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd278a1-ddf7-4794-ae4a-04ed81ca7915 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:48.562116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd278a1-ddf7-4794-ae4a-04ed81ca7915 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:48.562379 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd278a1-ddf7-4794-ae4a-04ed81ca7915 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:48.563078 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd278a1-ddf7-4794-ae4a-04ed81ca7915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:48.564687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd278a1-ddf7-4794-ae4a-04ed81ca7915 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:48.575305 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1291/2600] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:48.579716 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4394d55-08fd-47af-9787-1db593bed852 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:48.590737 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4394d55-08fd-47af-9787-1db593bed852 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:48.595435 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c0523-481c-4ecc-ae67-f4c898d69b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:48.601224 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c0523-481c-4ecc-ae67-f4c898d69b46 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:48.601224 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c0523-481c-4ecc-ae67-f4c898d69b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:48.601224 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c0523-481c-4ecc-ae67-f4c898d69b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:48.601224 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c0523-481c-4ecc-ae67-f4c898d69b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:48.601224 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c0523-481c-4ecc-ae67-f4c898d69b46 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:48.609545 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1310/2601] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:19:48.613878 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd278a1-ddf7-4794-ae4a-04ed81ca7915 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:48.624573 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd278a1-ddf7-4794-ae4a-04ed81ca7915 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:48.629229 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f7a63-c983-47f7-bbbc-923676e3cd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:48.638052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f7a63-c983-47f7-bbbc-923676e3cd88 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:48.638052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f7a63-c983-47f7-bbbc-923676e3cd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:48.638052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f7a63-c983-47f7-bbbc-923676e3cd88 admin admin] PATH_INFO: `/v3/projects/820240ce035d4f52bef45408e01b5698/users/2a49bbc125904d9d97689d7fa365ff3b/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:48.645566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f7a63-c983-47f7-bbbc-923676e3cd88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=820240ce035d4f52bef45408e01b5698, user_id=2a49bbc125904d9d97689d7fa365ff3b, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:48.648327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f7a63-c983-47f7-bbbc-923676e3cd88 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:48.727460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1292/2602] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:48 2022] PUT /identity/v3/projects/820240ce035d4f52bef45408e01b5698/users/2a49bbc125904d9d97689d7fa365ff3b/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:48.740304 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c1289-1c83-4e00-82a3-c16d7a7ff832 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:48.740682 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c1289-1c83-4e00-82a3-c16d7a7ff832 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:48.741040 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c1289-1c83-4e00-82a3-c16d7a7ff832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:48.774025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-1f9c1289-1c83-4e00-82a3-c16d7a7ff832 None None] MFA Rules not processed for user `2a49bbc125904d9d97689d7fa365ff3b`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:48.775223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-1f9c1289-1c83-4e00-82a3-c16d7a7ff832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:48.829244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:48.839583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1311/2603] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:48.844498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f7a63-c983-47f7-bbbc-923676e3cd88 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:48.890624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:48.910904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f7a63-c983-47f7-bbbc-923676e3cd88 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:48.921433 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba5de1-ce9c-4345-8b65-cc82f082bef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:48.922323 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba5de1-ce9c-4345-8b65-cc82f082bef4 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:48.922323 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba5de1-ce9c-4345-8b65-cc82f082bef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:48.922634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba5de1-ce9c-4345-8b65-cc82f082bef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:48.923240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba5de1-ce9c-4345-8b65-cc82f082bef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:48.925653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba5de1-ce9c-4345-8b65-cc82f082bef4 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:49.018036 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1293/2604] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:19:48 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:19:49.018327 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c1289-1c83-4e00-82a3-c16d7a7ff832 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:49.029318 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c1289-1c83-4e00-82a3-c16d7a7ff832 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:49.090042 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a93f28-6041-42f6-999d-9c27178a7290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:49.090679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a93f28-6041-42f6-999d-9c27178a7290 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:49.090957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a93f28-6041-42f6-999d-9c27178a7290 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:49.091223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a93f28-6041-42f6-999d-9c27178a7290 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:49.091851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a93f28-6041-42f6-999d-9c27178a7290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:49.094140 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a93f28-6041-42f6-999d-9c27178a7290 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:49.193149 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1312/2605] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:19:49 2022] POST /identity/v3/users => generated 294 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:19:49.197284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba5de1-ce9c-4345-8b65-cc82f082bef4 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:49.207739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba5de1-ce9c-4345-8b65-cc82f082bef4 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:49.212660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e9d15-f4eb-4acb-b125-22e7a8d086f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:49.213262 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e9d15-f4eb-4acb-b125-22e7a8d086f1 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:49.213543 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e9d15-f4eb-4acb-b125-22e7a8d086f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:49.213816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e9d15-f4eb-4acb-b125-22e7a8d086f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:49.214498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e9d15-f4eb-4acb-b125-22e7a8d086f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:49.216061 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e9d15-f4eb-4acb-b125-22e7a8d086f1 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:49.227036 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1294/2606] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:49.231416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a93f28-6041-42f6-999d-9c27178a7290 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:49.242361 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a93f28-6041-42f6-999d-9c27178a7290 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:49.248274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e30729-1ac6-4a42-95ad-b5b910c88b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:49.248934 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e30729-1ac6-4a42-95ad-b5b910c88b3c admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:49.249212 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e30729-1ac6-4a42-95ad-b5b910c88b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:49.249488 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e30729-1ac6-4a42-95ad-b5b910c88b3c admin admin] PATH_INFO: `/v3/projects/032a45820d56493da74b1bef28f688cc/users/d3496b4c24b34b96bddd130ca0f5c44b/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:49.264961 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e30729-1ac6-4a42-95ad-b5b910c88b3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=032a45820d56493da74b1bef28f688cc, user_id=d3496b4c24b34b96bddd130ca0f5c44b, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:49.270588 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e30729-1ac6-4a42-95ad-b5b910c88b3c admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:49.367106 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1313/2607] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:49 2022] PUT /identity/v3/projects/032a45820d56493da74b1bef28f688cc/users/d3496b4c24b34b96bddd130ca0f5c44b/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:49.372158 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e9d15-f4eb-4acb-b125-22e7a8d086f1 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:49.381742 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e9d15-f4eb-4acb-b125-22e7a8d086f1 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:49.442556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdde7f-0e6d-45b2-8a12-e6f0613c0c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:49.443170 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdde7f-0e6d-45b2-8a12-e6f0613c0c80 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:49.443449 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdde7f-0e6d-45b2-8a12-e6f0613c0c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:49.443721 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdde7f-0e6d-45b2-8a12-e6f0613c0c80 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:49.444439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdde7f-0e6d-45b2-8a12-e6f0613c0c80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:49.446141 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdde7f-0e6d-45b2-8a12-e6f0613c0c80 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:49.459710 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1295/2608] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:49.461773 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e30729-1ac6-4a42-95ad-b5b910c88b3c admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:49.472049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e30729-1ac6-4a42-95ad-b5b910c88b3c admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:49.477500 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fdd538-3980-43aa-a865-00abc5f88390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:49.478150 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fdd538-3980-43aa-a865-00abc5f88390 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:49.478427 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fdd538-3980-43aa-a865-00abc5f88390 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:49.478696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fdd538-3980-43aa-a865-00abc5f88390 admin admin] PATH_INFO: `/v3/projects/032a45820d56493da74b1bef28f688cc/users/d3496b4c24b34b96bddd130ca0f5c44b/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:49.480860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fdd538-3980-43aa-a865-00abc5f88390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=032a45820d56493da74b1bef28f688cc, user_id=d3496b4c24b34b96bddd130ca0f5c44b, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:49.483360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fdd538-3980-43aa-a865-00abc5f88390 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:49.494024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1314/2609] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:49 2022] PUT /identity/v3/projects/032a45820d56493da74b1bef28f688cc/users/d3496b4c24b34b96bddd130ca0f5c44b/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:49.505820 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86fd58-0d93-4b18-b894-f58cc9b8aeac None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:49.506112 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86fd58-0d93-4b18-b894-f58cc9b8aeac None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:49.506391 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86fd58-0d93-4b18-b894-f58cc9b8aeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:49.535742 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-3b86fd58-0d93-4b18-b894-f58cc9b8aeac None None] MFA Rules not processed for user `d3496b4c24b34b96bddd130ca0f5c44b`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:49.537048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3b86fd58-0d93-4b18-b894-f58cc9b8aeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:49.609291 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1296/2610] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:49.621033 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fdd538-3980-43aa-a865-00abc5f88390 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:49.622745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-59fdd538-3980-43aa-a865-00abc5f88390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:49.684787 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fdd538-3980-43aa-a865-00abc5f88390 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:49.688837 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2b54c-7f2d-4d88-b53c-e1aed952d01d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:49.689620 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2b54c-7f2d-4d88-b53c-e1aed952d01d service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:49.689893 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2b54c-7f2d-4d88-b53c-e1aed952d01d service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:49.690297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2b54c-7f2d-4d88-b53c-e1aed952d01d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:49.751290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2b54c-7f2d-4d88-b53c-e1aed952d01d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:49.752749 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2b54c-7f2d-4d88-b53c-e1aed952d01d service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:49.758820 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:49.763313 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1315/2611] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:19:55.098371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86fd58-0d93-4b18-b894-f58cc9b8aeac None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:55.142188 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:55.186067 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86fd58-0d93-4b18-b894-f58cc9b8aeac None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:55.190980 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de807b-9a14-4b16-b071-0c3ff99ec9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:55.191670 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de807b-9a14-4b16-b071-0c3ff99ec9e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:55.191949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de807b-9a14-4b16-b071-0c3ff99ec9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:55.192243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de807b-9a14-4b16-b071-0c3ff99ec9e3 admin admin] PATH_INFO: `/v3/users/2a49bbc125904d9d97689d7fa365ff3b` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:55.193484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de807b-9a14-4b16-b071-0c3ff99ec9e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a49bbc125904d9d97689d7fa365ff3b)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:55.195503 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de807b-9a14-4b16-b071-0c3ff99ec9e3 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:55.375141 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-03de807b-9a14-4b16-b071-0c3ff99ec9e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a49bbc125904d9d97689d7fa365ff3b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:55.384190 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-03de807b-9a14-4b16-b071-0c3ff99ec9e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a49bbc125904d9d97689d7fa365ff3b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:55.384906 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-03de807b-9a14-4b16-b071-0c3ff99ec9e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a49bbc125904d9d97689d7fa365ff3b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:55.398439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-03de807b-9a14-4b16-b071-0c3ff99ec9e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a49bbc125904d9d97689d7fa365ff3b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:55.399461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-03de807b-9a14-4b16-b071-0c3ff99ec9e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a49bbc125904d9d97689d7fa365ff3b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:55.494461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1297/2612] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:19:55 2022] DELETE /identity/v3/users/2a49bbc125904d9d97689d7fa365ff3b => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:55.499046 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10908f5-9ed6-49ba-8129-b4c68f5f6ec1 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:55.499576 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10908f5-9ed6-49ba-8129-b4c68f5f6ec1 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:55.499576 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10908f5-9ed6-49ba-8129-b4c68f5f6ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:55.519045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-f10908f5-9ed6-49ba-8129-b4c68f5f6ec1 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:55.520317 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f10908f5-9ed6-49ba-8129-b4c68f5f6ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:55.588055 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1316/2613] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:55.598213 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de807b-9a14-4b16-b071-0c3ff99ec9e3 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:55.599759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-03de807b-9a14-4b16-b071-0c3ff99ec9e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:55.657874 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de807b-9a14-4b16-b071-0c3ff99ec9e3 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:55.661453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e1ccb4-dab2-42d3-b433-d14361e6e06e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:55.662050 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1ccb4-dab2-42d3-b433-d14361e6e06e service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:55.662322 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1ccb4-dab2-42d3-b433-d14361e6e06e service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:55.662588 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1ccb4-dab2-42d3-b433-d14361e6e06e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:55.672247 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e1ccb4-dab2-42d3-b433-d14361e6e06e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:55.673651 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e1ccb4-dab2-42d3-b433-d14361e6e06e service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:55.683255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1298/2614] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:19:56.128382 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10908f5-9ed6-49ba-8129-b4c68f5f6ec1 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:56.129813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f10908f5-9ed6-49ba-8129-b4c68f5f6ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:56.142906 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10908f5-9ed6-49ba-8129-b4c68f5f6ec1 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:56.148072 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3e650-ba35-470e-8a38-25b40cac85fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:56.148693 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3e650-ba35-470e-8a38-25b40cac85fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:56.148968 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3e650-ba35-470e-8a38-25b40cac85fd admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:56.149234 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3e650-ba35-470e-8a38-25b40cac85fd admin admin] PATH_INFO: `/v3/projects/820240ce035d4f52bef45408e01b5698` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:56.150370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3e650-ba35-470e-8a38-25b40cac85fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=820240ce035d4f52bef45408e01b5698)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:56.152343 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3e650-ba35-470e-8a38-25b40cac85fd admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:56.261437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-2cf3e650-ba35-470e-8a38-25b40cac85fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '820240ce035d4f52bef45408e01b5698', 'request_id': 'req-2cf3e650-ba35-470e-8a38-25b40cac85fd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:56.268047 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-2cf3e650-ba35-470e-8a38-25b40cac85fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '820240ce035d4f52bef45408e01b5698', 'request_id': 'req-2cf3e650-ba35-470e-8a38-25b40cac85fd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:56.274660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-2cf3e650-ba35-470e-8a38-25b40cac85fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '820240ce035d4f52bef45408e01b5698', 'request_id': 'req-2cf3e650-ba35-470e-8a38-25b40cac85fd'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:56.369340 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-2cf3e650-ba35-470e-8a38-25b40cac85fd admin admin] The token cache is being invalidate because project 820240ce035d4f52bef45408e01b5698 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:56.369944 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-2cf3e650-ba35-470e-8a38-25b40cac85fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:56.370673 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-2cf3e650-ba35-470e-8a38-25b40cac85fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:56.375303 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1317/2615] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:56 2022] DELETE /identity/v3/projects/820240ce035d4f52bef45408e01b5698 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:56.380565 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e1ccb4-dab2-42d3-b433-d14361e6e06e service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:56.389256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e1ccb4-dab2-42d3-b433-d14361e6e06e service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:56.450427 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:56.456656 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5f500-d618-4b2c-95cd-51232562c883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:56.457294 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5f500-d618-4b2c-95cd-51232562c883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:56.457572 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5f500-d618-4b2c-95cd-51232562c883 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:56.457843 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5f500-d618-4b2c-95cd-51232562c883 admin admin] PATH_INFO: `/v3/users/d3496b4c24b34b96bddd130ca0f5c44b` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:56.459036 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5f500-d618-4b2c-95cd-51232562c883 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3496b4c24b34b96bddd130ca0f5c44b)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:56.461523 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5f500-d618-4b2c-95cd-51232562c883 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:56.622177 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c2a5f500-d618-4b2c-95cd-51232562c883 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3496b4c24b34b96bddd130ca0f5c44b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:56.629686 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c2a5f500-d618-4b2c-95cd-51232562c883 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3496b4c24b34b96bddd130ca0f5c44b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:56.630487 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c2a5f500-d618-4b2c-95cd-51232562c883 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3496b4c24b34b96bddd130ca0f5c44b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:56.644383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c2a5f500-d618-4b2c-95cd-51232562c883 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3496b4c24b34b96bddd130ca0f5c44b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:56.647758 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c2a5f500-d618-4b2c-95cd-51232562c883 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3496b4c24b34b96bddd130ca0f5c44b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:56.740514 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1299/2616] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:19:56 2022] DELETE /identity/v3/users/d3496b4c24b34b96bddd130ca0f5c44b => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:57.210751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3e650-ba35-470e-8a38-25b40cac85fd admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:57.212244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-2cf3e650-ba35-470e-8a38-25b40cac85fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:57.226789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3e650-ba35-470e-8a38-25b40cac85fd admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:57.286386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf58c9b-7486-4705-abf4-0170a15f5a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:57.286970 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf58c9b-7486-4705-abf4-0170a15f5a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:57.287245 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf58c9b-7486-4705-abf4-0170a15f5a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:57.287516 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf58c9b-7486-4705-abf4-0170a15f5a7f admin admin] PATH_INFO: `/v3/projects/032a45820d56493da74b1bef28f688cc` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:57.289219 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf58c9b-7486-4705-abf4-0170a15f5a7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=032a45820d56493da74b1bef28f688cc)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:57.291228 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf58c9b-7486-4705-abf4-0170a15f5a7f admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:57.393469 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:57.411788 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8cf58c9b-7486-4705-abf4-0170a15f5a7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '032a45820d56493da74b1bef28f688cc', 'request_id': 'req-8cf58c9b-7486-4705-abf4-0170a15f5a7f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:57.418575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8cf58c9b-7486-4705-abf4-0170a15f5a7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '032a45820d56493da74b1bef28f688cc', 'request_id': 'req-8cf58c9b-7486-4705-abf4-0170a15f5a7f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:57.424699 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8cf58c9b-7486-4705-abf4-0170a15f5a7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '032a45820d56493da74b1bef28f688cc', 'request_id': 'req-8cf58c9b-7486-4705-abf4-0170a15f5a7f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:57.572952 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8cf58c9b-7486-4705-abf4-0170a15f5a7f admin admin] The token cache is being invalidate because project 032a45820d56493da74b1bef28f688cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:19:57.573165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8cf58c9b-7486-4705-abf4-0170a15f5a7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:57.573780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8cf58c9b-7486-4705-abf4-0170a15f5a7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:19:57.575283 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1318/2617] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:19:57 2022] DELETE /identity/v3/projects/032a45820d56493da74b1bef28f688cc => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:57.580514 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634852cf-8355-4663-96c4-f604e4aa87fc None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:57.581004 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634852cf-8355-4663-96c4-f604e4aa87fc None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:57.581123 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634852cf-8355-4663-96c4-f604e4aa87fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:57.601657 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-634852cf-8355-4663-96c4-f604e4aa87fc None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:57.605092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-634852cf-8355-4663-96c4-f604e4aa87fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:57.664505 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1300/2618] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:57.672371 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745a0cd-81a6-407e-a5e1-eb753d73e7b9 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:57.672649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745a0cd-81a6-407e-a5e1-eb753d73e7b9 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:57.672916 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745a0cd-81a6-407e-a5e1-eb753d73e7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:57.694737 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-4745a0cd-81a6-407e-a5e1-eb753d73e7b9 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:57.695947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-4745a0cd-81a6-407e-a5e1-eb753d73e7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:57.706133 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1319/2619] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:19:57.710435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634852cf-8355-4663-96c4-f604e4aa87fc None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:57.721185 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634852cf-8355-4663-96c4-f604e4aa87fc None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:57.725978 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34693a55-a4a9-4ff9-91fb-b8e88f5f7eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:57.726575 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34693a55-a4a9-4ff9-91fb-b8e88f5f7eec admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:57.726851 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34693a55-a4a9-4ff9-91fb-b8e88f5f7eec admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:57.727118 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34693a55-a4a9-4ff9-91fb-b8e88f5f7eec admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:57.728672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34693a55-a4a9-4ff9-91fb-b8e88f5f7eec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:19:57.729006 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34693a55-a4a9-4ff9-91fb-b8e88f5f7eec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:57.730584 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34693a55-a4a9-4ff9-91fb-b8e88f5f7eec admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:57.743079 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1301/2620] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:19:57.750620 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745a0cd-81a6-407e-a5e1-eb753d73e7b9 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:57.760957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745a0cd-81a6-407e-a5e1-eb753d73e7b9 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:57.770033 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160d0b7-edea-49cd-a91a-ee599043aaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:57.770651 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160d0b7-edea-49cd-a91a-ee599043aaa6 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:57.772268 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160d0b7-edea-49cd-a91a-ee599043aaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:57.772268 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160d0b7-edea-49cd-a91a-ee599043aaa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:57.772268 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160d0b7-edea-49cd-a91a-ee599043aaa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:57.773828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160d0b7-edea-49cd-a91a-ee599043aaa6 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:57.903046 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1320/2621] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:19:57 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:19:57.907206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34693a55-a4a9-4ff9-91fb-b8e88f5f7eec admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:57.917916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34693a55-a4a9-4ff9-91fb-b8e88f5f7eec admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:57.979097 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0125b-1baa-4617-b54f-b508eda71905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:57.979693 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0125b-1baa-4617-b54f-b508eda71905 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:57.979966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0125b-1baa-4617-b54f-b508eda71905 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:57.980263 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0125b-1baa-4617-b54f-b508eda71905 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:57.980875 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0125b-1baa-4617-b54f-b508eda71905 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:57.982768 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0125b-1baa-4617-b54f-b508eda71905 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:57.993040 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:19:58.117372 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1302/2622] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:19:57 2022] POST /identity/v3/users => generated 292 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:19:58.125583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160d0b7-edea-49cd-a91a-ee599043aaa6 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:58.136490 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160d0b7-edea-49cd-a91a-ee599043aaa6 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:58.141215 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3761690-8d0c-4bdc-a156-317e11a133b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:58.141816 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3761690-8d0c-4bdc-a156-317e11a133b3 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:58.142093 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3761690-8d0c-4bdc-a156-317e11a133b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:58.142363 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3761690-8d0c-4bdc-a156-317e11a133b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:58.143003 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3761690-8d0c-4bdc-a156-317e11a133b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:58.144615 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3761690-8d0c-4bdc-a156-317e11a133b3 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:58.155751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1321/2623] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:58.159885 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0125b-1baa-4617-b54f-b508eda71905 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:58.171518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0125b-1baa-4617-b54f-b508eda71905 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:58.176958 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f721c28-5179-4a9d-b83d-e2e5f80d0dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:58.177560 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f721c28-5179-4a9d-b83d-e2e5f80d0dab admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:58.177840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f721c28-5179-4a9d-b83d-e2e5f80d0dab admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:58.178111 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f721c28-5179-4a9d-b83d-e2e5f80d0dab admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:58.178750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f721c28-5179-4a9d-b83d-e2e5f80d0dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:58.180384 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f721c28-5179-4a9d-b83d-e2e5f80d0dab admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:58.191679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1303/2624] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:19:58.196252 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3761690-8d0c-4bdc-a156-317e11a133b3 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:58.206942 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3761690-8d0c-4bdc-a156-317e11a133b3 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:58.211581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b50f06-bb7e-4368-bdc6-aa76ee0ecc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:58.212796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b50f06-bb7e-4368-bdc6-aa76ee0ecc25 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:58.213079 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b50f06-bb7e-4368-bdc6-aa76ee0ecc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:58.213348 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b50f06-bb7e-4368-bdc6-aa76ee0ecc25 admin admin] PATH_INFO: `/v3/projects/8b61dd30bd20426990ed70288f69e40b/users/79a2748b61b4451f9215a4f8eea5dea2/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:58.228906 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b50f06-bb7e-4368-bdc6-aa76ee0ecc25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b61dd30bd20426990ed70288f69e40b, user_id=79a2748b61b4451f9215a4f8eea5dea2, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:58.232996 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b50f06-bb7e-4368-bdc6-aa76ee0ecc25 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:58.338816 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1322/2625] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:19:58 2022] PUT /identity/v3/projects/8b61dd30bd20426990ed70288f69e40b/users/79a2748b61b4451f9215a4f8eea5dea2/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:19:58.343973 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d74ffe-481a-4ddb-97cb-d0b768e54c93 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:58.344290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d74ffe-481a-4ddb-97cb-d0b768e54c93 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:58.344560 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d74ffe-481a-4ddb-97cb-d0b768e54c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:58.363831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-57d74ffe-481a-4ddb-97cb-d0b768e54c93 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:19:58.365060 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-57d74ffe-481a-4ddb-97cb-d0b768e54c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:58.440054 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1304/2626] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:19:58.450592 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b50f06-bb7e-4368-bdc6-aa76ee0ecc25 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:19:58.452451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-86b50f06-bb7e-4368-bdc6-aa76ee0ecc25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:19:58.508940 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b50f06-bb7e-4368-bdc6-aa76ee0ecc25 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:19:58.512765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddce350-981a-4416-b058-1602bcf2bab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:19:58.513363 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddce350-981a-4416-b058-1602bcf2bab7 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:19:58.513646 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddce350-981a-4416-b058-1602bcf2bab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:19:58.513911 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddce350-981a-4416-b058-1602bcf2bab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:19:58.522906 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddce350-981a-4416-b058-1602bcf2bab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:19:58.524409 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddce350-981a-4416-b058-1602bcf2bab7 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:19:58.535865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1323/2627] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:19:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:20:04.330025 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046f74e-7267-4d90-a9e6-a05c2f8d25c2 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:04.330612 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046f74e-7267-4d90-a9e6-a05c2f8d25c2 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:04.330612 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046f74e-7267-4d90-a9e6-a05c2f8d25c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:04.368345 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-9046f74e-7267-4d90-a9e6-a05c2f8d25c2 None None] MFA Rules not processed for user `79a2748b61b4451f9215a4f8eea5dea2`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:04.369694 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-9046f74e-7267-4d90-a9e6-a05c2f8d25c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:04.467305 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1305/2628] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:04.477679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddce350-981a-4416-b058-1602bcf2bab7 service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:04.489536 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddce350-981a-4416-b058-1602bcf2bab7 service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:04.495081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb694213-c738-4b4f-8fb3-abf2439a1e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:04.495707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb694213-c738-4b4f-8fb3-abf2439a1e30 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:04.495988 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb694213-c738-4b4f-8fb3-abf2439a1e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:04.496303 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb694213-c738-4b4f-8fb3-abf2439a1e30 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:04.496979 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb694213-c738-4b4f-8fb3-abf2439a1e30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:04.498926 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb694213-c738-4b4f-8fb3-abf2439a1e30 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:04.600434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1324/2629] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:20:04 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:20:04.606211 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046f74e-7267-4d90-a9e6-a05c2f8d25c2 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:04.616949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046f74e-7267-4d90-a9e6-a05c2f8d25c2 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:04.674907 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88f6ec-ee32-41ce-baa7-b29530a8e6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:04.675523 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88f6ec-ee32-41ce-baa7-b29530a8e6eb admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:04.675804 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88f6ec-ee32-41ce-baa7-b29530a8e6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:04.676102 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88f6ec-ee32-41ce-baa7-b29530a8e6eb admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:04.676760 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88f6ec-ee32-41ce-baa7-b29530a8e6eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:04.678773 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88f6ec-ee32-41ce-baa7-b29530a8e6eb admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:04.809901 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1306/2630] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:20:04 2022] POST /identity/v3/users => generated 299 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:20:04.814204 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb694213-c738-4b4f-8fb3-abf2439a1e30 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:04.824745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb694213-c738-4b4f-8fb3-abf2439a1e30 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:04.829330 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4caa99-4ae8-4f23-a6b0-c6db6a775da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:04.830032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4caa99-4ae8-4f23-a6b0-c6db6a775da1 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:04.830261 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4caa99-4ae8-4f23-a6b0-c6db6a775da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:04.830531 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4caa99-4ae8-4f23-a6b0-c6db6a775da1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:04.831197 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4caa99-4ae8-4f23-a6b0-c6db6a775da1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:04.832925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4caa99-4ae8-4f23-a6b0-c6db6a775da1 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:04.844110 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1325/2631] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:20:04.848888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88f6ec-ee32-41ce-baa7-b29530a8e6eb admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:04.854711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:04.860035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88f6ec-ee32-41ce-baa7-b29530a8e6eb admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:04.864633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c220dc-283c-4467-88d8-db801738d440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:04.865238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c220dc-283c-4467-88d8-db801738d440 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:04.865514 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c220dc-283c-4467-88d8-db801738d440 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:04.865779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c220dc-283c-4467-88d8-db801738d440 admin admin] PATH_INFO: `/v3/projects/321e257d63514d289308ea1b6b257528/users/fc466057f33f409a9d5aa5dfbee8269c/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:04.881045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c220dc-283c-4467-88d8-db801738d440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=321e257d63514d289308ea1b6b257528, user_id=fc466057f33f409a9d5aa5dfbee8269c, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:04.883670 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c220dc-283c-4467-88d8-db801738d440 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:04.974080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1307/2632] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:20:04 2022] PUT /identity/v3/projects/321e257d63514d289308ea1b6b257528/users/fc466057f33f409a9d5aa5dfbee8269c/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:04.980309 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4caa99-4ae8-4f23-a6b0-c6db6a775da1 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:04.990823 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4caa99-4ae8-4f23-a6b0-c6db6a775da1 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:05.009227 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:05.063699 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea805b-3029-4725-ad02-0f5d3edb1b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:05.064514 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea805b-3029-4725-ad02-0f5d3edb1b48 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:05.065188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea805b-3029-4725-ad02-0f5d3edb1b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:05.065790 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea805b-3029-4725-ad02-0f5d3edb1b48 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:05.066987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea805b-3029-4725-ad02-0f5d3edb1b48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:05.068898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea805b-3029-4725-ad02-0f5d3edb1b48 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:05.084832 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1326/2633] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:20:05.086870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c220dc-283c-4467-88d8-db801738d440 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:05.100912 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c220dc-283c-4467-88d8-db801738d440 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:05.110862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226cbaf-ee1c-4a8d-9236-68c742153654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:05.111639 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226cbaf-ee1c-4a8d-9236-68c742153654 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:05.112186 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226cbaf-ee1c-4a8d-9236-68c742153654 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:05.113938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226cbaf-ee1c-4a8d-9236-68c742153654 admin admin] PATH_INFO: `/v3/projects/321e257d63514d289308ea1b6b257528/users/fc466057f33f409a9d5aa5dfbee8269c/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:05.114608 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0226cbaf-ee1c-4a8d-9236-68c742153654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=321e257d63514d289308ea1b6b257528, user_id=fc466057f33f409a9d5aa5dfbee8269c, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:05.117761 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0226cbaf-ee1c-4a8d-9236-68c742153654 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:05.136571 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1308/2634] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:20:05 2022] PUT /identity/v3/projects/321e257d63514d289308ea1b6b257528/users/fc466057f33f409a9d5aa5dfbee8269c/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:06.371969 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea805b-3029-4725-ad02-0f5d3edb1b48 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:06.381455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-16ea805b-3029-4725-ad02-0f5d3edb1b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:06.386932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea805b-3029-4725-ad02-0f5d3edb1b48 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:06.450392 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd02ec-9ca0-4c34-bf86-aa99eb01ef20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:06.451025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd02ec-9ca0-4c34-bf86-aa99eb01ef20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:06.451298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd02ec-9ca0-4c34-bf86-aa99eb01ef20 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:06.451569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd02ec-9ca0-4c34-bf86-aa99eb01ef20 admin admin] PATH_INFO: `/v3/users/b872b378c222451b97e3483bc8b3d766` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:06.452880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd02ec-9ca0-4c34-bf86-aa99eb01ef20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b872b378c222451b97e3483bc8b3d766)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:06.454769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd02ec-9ca0-4c34-bf86-aa99eb01ef20 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:06.633838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6efd02ec-9ca0-4c34-bf86-aa99eb01ef20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b872b378c222451b97e3483bc8b3d766'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:06.636493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6efd02ec-9ca0-4c34-bf86-aa99eb01ef20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b872b378c222451b97e3483bc8b3d766'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:06.647986 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6efd02ec-9ca0-4c34-bf86-aa99eb01ef20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b872b378c222451b97e3483bc8b3d766'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:06.716049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6efd02ec-9ca0-4c34-bf86-aa99eb01ef20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b872b378c222451b97e3483bc8b3d766'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:06.719846 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6efd02ec-9ca0-4c34-bf86-aa99eb01ef20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b872b378c222451b97e3483bc8b3d766'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:06.743183 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1327/2635] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:20:06 2022] DELETE /identity/v3/users/b872b378c222451b97e3483bc8b3d766 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:06.748267 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89208369-a73b-4e18-ac23-2e547d3fa751 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:06.748785 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89208369-a73b-4e18-ac23-2e547d3fa751 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:06.748785 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89208369-a73b-4e18-ac23-2e547d3fa751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:06.775045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-89208369-a73b-4e18-ac23-2e547d3fa751 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:06.776682 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-89208369-a73b-4e18-ac23-2e547d3fa751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:06.887903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1309/2636] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:06.898899 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd02ec-9ca0-4c34-bf86-aa99eb01ef20 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:06.901400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6efd02ec-9ca0-4c34-bf86-aa99eb01ef20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:06.953163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:06.982132 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd02ec-9ca0-4c34-bf86-aa99eb01ef20 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:06.991873 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04294a6e-4940-4689-a1bc-0648b953086e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:06.992824 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04294a6e-4940-4689-a1bc-0648b953086e service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:06.993110 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04294a6e-4940-4689-a1bc-0648b953086e service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:06.993465 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04294a6e-4940-4689-a1bc-0648b953086e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:07.054393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04294a6e-4940-4689-a1bc-0648b953086e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:07.071774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04294a6e-4940-4689-a1bc-0648b953086e service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:07.090747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1328/2637] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:20:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:20:07.885812 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89208369-a73b-4e18-ac23-2e547d3fa751 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:07.885812 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-89208369-a73b-4e18-ac23-2e547d3fa751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:07.910327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89208369-a73b-4e18-ac23-2e547d3fa751 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:07.918112 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90b00b-466c-41fb-b7a4-f1f04626ac74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:07.919142 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90b00b-466c-41fb-b7a4-f1f04626ac74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:07.919589 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90b00b-466c-41fb-b7a4-f1f04626ac74 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:07.920059 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90b00b-466c-41fb-b7a4-f1f04626ac74 admin admin] PATH_INFO: `/v3/projects/8d1d8bd83b2247bb9f59f1cf6d695fbf` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:07.923969 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af90b00b-466c-41fb-b7a4-f1f04626ac74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d1d8bd83b2247bb9f59f1cf6d695fbf)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:07.926122 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af90b00b-466c-41fb-b7a4-f1f04626ac74 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:08.044284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-af90b00b-466c-41fb-b7a4-f1f04626ac74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d1d8bd83b2247bb9f59f1cf6d695fbf', 'request_id': 'req-af90b00b-466c-41fb-b7a4-f1f04626ac74'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:08.117790 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-af90b00b-466c-41fb-b7a4-f1f04626ac74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d1d8bd83b2247bb9f59f1cf6d695fbf', 'request_id': 'req-af90b00b-466c-41fb-b7a4-f1f04626ac74'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:08.128840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-af90b00b-466c-41fb-b7a4-f1f04626ac74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d1d8bd83b2247bb9f59f1cf6d695fbf', 'request_id': 'req-af90b00b-466c-41fb-b7a4-f1f04626ac74'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:08.144664 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-af90b00b-466c-41fb-b7a4-f1f04626ac74 admin admin] The token cache is being invalidate because project 8d1d8bd83b2247bb9f59f1cf6d695fbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:20:08.145338 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-af90b00b-466c-41fb-b7a4-f1f04626ac74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:08.146272 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-af90b00b-466c-41fb-b7a4-f1f04626ac74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:08.150372 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1310/2638] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:20:07 2022] DELETE /identity/v3/projects/8d1d8bd83b2247bb9f59f1cf6d695fbf => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:08.155297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722b3f9-1511-4c29-9aad-78579ee3a8d0 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:08.156018 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722b3f9-1511-4c29-9aad-78579ee3a8d0 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:08.156159 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722b3f9-1511-4c29-9aad-78579ee3a8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:08.183475 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-e722b3f9-1511-4c29-9aad-78579ee3a8d0 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:08.184780 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e722b3f9-1511-4c29-9aad-78579ee3a8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:08.271767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1329/2639] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:20:08.280893 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53868ea-92f6-4206-89b4-701205ca5c9f None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:08.281492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53868ea-92f6-4206-89b4-701205ca5c9f None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:08.281835 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53868ea-92f6-4206-89b4-701205ca5c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:08.307496 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-c53868ea-92f6-4206-89b4-701205ca5c9f None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:08.308778 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-c53868ea-92f6-4206-89b4-701205ca5c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:08.320578 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1311/2640] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:08.325258 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722b3f9-1511-4c29-9aad-78579ee3a8d0 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:08.340416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722b3f9-1511-4c29-9aad-78579ee3a8d0 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:08.349002 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3187d1f-ab23-4275-a0ae-fe4d51f0d7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:08.358324 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3187d1f-ab23-4275-a0ae-fe4d51f0d7c9 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:08.358324 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3187d1f-ab23-4275-a0ae-fe4d51f0d7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:08.358324 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3187d1f-ab23-4275-a0ae-fe4d51f0d7c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:08.358324 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3187d1f-ab23-4275-a0ae-fe4d51f0d7c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:20:08.358324 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3187d1f-ab23-4275-a0ae-fe4d51f0d7c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:08.362896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3187d1f-ab23-4275-a0ae-fe4d51f0d7c9 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:08.389106 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1330/2641] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:20:08.390264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53868ea-92f6-4206-89b4-701205ca5c9f None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:08.404439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53868ea-92f6-4206-89b4-701205ca5c9f None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:08.418307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2b60c-696e-4f90-86d7-030ee8376c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:08.418947 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2b60c-696e-4f90-86d7-030ee8376c87 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:08.424163 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2b60c-696e-4f90-86d7-030ee8376c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:08.424163 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2b60c-696e-4f90-86d7-030ee8376c87 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:08.424163 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2b60c-696e-4f90-86d7-030ee8376c87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:08.425364 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2b60c-696e-4f90-86d7-030ee8376c87 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:08.532593 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1312/2642] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:20:08 2022] POST /identity/v3/projects => generated 371 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:20:08.534030 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3187d1f-ab23-4275-a0ae-fe4d51f0d7c9 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:08.545812 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3187d1f-ab23-4275-a0ae-fe4d51f0d7c9 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:08.616263 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a86c2-0506-4860-9114-e7d0c594e1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:08.617025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a86c2-0506-4860-9114-e7d0c594e1e8 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:08.617455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a86c2-0506-4860-9114-e7d0c594e1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:08.617725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a86c2-0506-4860-9114-e7d0c594e1e8 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:08.618680 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a86c2-0506-4860-9114-e7d0c594e1e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:08.620729 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a86c2-0506-4860-9114-e7d0c594e1e8 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:08.747703 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1331/2643] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:20:08 2022] POST /identity/v3/users => generated 286 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:20:08.749175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2b60c-696e-4f90-86d7-030ee8376c87 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:08.763142 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2b60c-696e-4f90-86d7-030ee8376c87 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:08.769237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923eca6-f3e8-4f01-8e7f-cbfd18d8e061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:08.773400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923eca6-f3e8-4f01-8e7f-cbfd18d8e061 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:08.773400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923eca6-f3e8-4f01-8e7f-cbfd18d8e061 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:08.773400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923eca6-f3e8-4f01-8e7f-cbfd18d8e061 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:08.773400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923eca6-f3e8-4f01-8e7f-cbfd18d8e061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:08.773400 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923eca6-f3e8-4f01-8e7f-cbfd18d8e061 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:08.787764 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1313/2644] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:20:08.789438 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a86c2-0506-4860-9114-e7d0c594e1e8 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:08.800929 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a86c2-0506-4860-9114-e7d0c594e1e8 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:08.806962 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b69546-14b8-4ec5-9351-9ace602c2e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:08.807584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b69546-14b8-4ec5-9351-9ace602c2e5e admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:08.807864 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b69546-14b8-4ec5-9351-9ace602c2e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:08.808160 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b69546-14b8-4ec5-9351-9ace602c2e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:08.808816 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b69546-14b8-4ec5-9351-9ace602c2e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:08.810443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b69546-14b8-4ec5-9351-9ace602c2e5e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:08.822843 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1332/2645] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:20:08.827200 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923eca6-f3e8-4f01-8e7f-cbfd18d8e061 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:08.848076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923eca6-f3e8-4f01-8e7f-cbfd18d8e061 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:08.850476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2152f07f-7537-4462-8524-6b205100c1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:08.851247 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152f07f-7537-4462-8524-6b205100c1dd admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:08.851654 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152f07f-7537-4462-8524-6b205100c1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:08.853575 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152f07f-7537-4462-8524-6b205100c1dd admin admin] PATH_INFO: `/v3/projects/a34d8e04191a4b28b7fb3e19ed43b1d8/users/79b68aa949d24074be074ef82a541bc5/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:08.856506 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:08.870408 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2152f07f-7537-4462-8524-6b205100c1dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a34d8e04191a4b28b7fb3e19ed43b1d8, user_id=79b68aa949d24074be074ef82a541bc5, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:08.873393 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2152f07f-7537-4462-8524-6b205100c1dd admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:08.976046 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1314/2646] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:20:08 2022] PUT /identity/v3/projects/a34d8e04191a4b28b7fb3e19ed43b1d8/users/79b68aa949d24074be074ef82a541bc5/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:08.983539 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b8a9d-40eb-4afa-a4de-ec0736fb49ed None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:08.983539 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b8a9d-40eb-4afa-a4de-ec0736fb49ed None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:08.983539 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b8a9d-40eb-4afa-a4de-ec0736fb49ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:09.031283 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-222b8a9d-40eb-4afa-a4de-ec0736fb49ed None None] MFA Rules not processed for user `79b68aa949d24074be074ef82a541bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:09.036847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-222b8a9d-40eb-4afa-a4de-ec0736fb49ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:09.108890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:09.174435 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1333/2647] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:09.186621 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2152f07f-7537-4462-8524-6b205100c1dd admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:09.189478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-2152f07f-7537-4462-8524-6b205100c1dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:09.263697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2152f07f-7537-4462-8524-6b205100c1dd admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:09.267916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11012bd7-78bb-4fd6-9054-d035a8cd3e27 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4db08aa2e4ef4478a3909a876754814c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:09.269151 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11012bd7-78bb-4fd6-9054-d035a8cd3e27 service glance] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:09.269609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11012bd7-78bb-4fd6-9054-d035a8cd3e27 service glance] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:09.270058 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11012bd7-78bb-4fd6-9054-d035a8cd3e27 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:09.281198 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11012bd7-78bb-4fd6-9054-d035a8cd3e27 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:09.282644 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11012bd7-78bb-4fd6-9054-d035a8cd3e27 service glance] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:09.294058 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1315/2648] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:20:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:20:11.327287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176a92e-b048-47cc-be73-ef8706f0160f None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:11.327803 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176a92e-b048-47cc-be73-ef8706f0160f None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:11.327911 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176a92e-b048-47cc-be73-ef8706f0160f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:11.381146 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-5176a92e-b048-47cc-be73-ef8706f0160f None None] MFA Rules not processed for user `fc466057f33f409a9d5aa5dfbee8269c`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:11.382538 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-5176a92e-b048-47cc-be73-ef8706f0160f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:11.462908 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1334/2649] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:11.478584 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11012bd7-78bb-4fd6-9054-d035a8cd3e27 service glance] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:11.488889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11012bd7-78bb-4fd6-9054-d035a8cd3e27 service glance] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:11.492633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457bf7ab-f4bb-4dce-bda2-8d759e6d0914 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4db08aa2e4ef4478a3909a876754814c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:11.493238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bf7ab-f4bb-4dce-bda2-8d759e6d0914 service glance] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:11.493708 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bf7ab-f4bb-4dce-bda2-8d759e6d0914 service glance] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:11.493984 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bf7ab-f4bb-4dce-bda2-8d759e6d0914 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:11.495782 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-457bf7ab-f4bb-4dce-bda2-8d759e6d0914 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:11.584602 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457bf7ab-f4bb-4dce-bda2-8d759e6d0914 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:11.586117 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457bf7ab-f4bb-4dce-bda2-8d759e6d0914 service glance] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:11.596837 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1316/2650] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:20:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:20:11.821573 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0dd0c-57e9-4b55-855d-6bfc186b99ab None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:11.822752 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0dd0c-57e9-4b55-855d-6bfc186b99ab None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:11.823049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0dd0c-57e9-4b55-855d-6bfc186b99ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:11.891009 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-1fd0dd0c-57e9-4b55-855d-6bfc186b99ab None None] MFA Rules not processed for user `980157efae8f43b59dc063fda04c0348`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:11.892327 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-1fd0dd0c-57e9-4b55-855d-6bfc186b99ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:11.968021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1335/2651] 10.222.0.59 () {58 vars in 1009 bytes} [Mon Aug 29 23:20:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:12.103738 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457bf7ab-f4bb-4dce-bda2-8d759e6d0914 service glance] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:12.105206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-457bf7ab-f4bb-4dce-bda2-8d759e6d0914 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:12.199082 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457bf7ab-f4bb-4dce-bda2-8d759e6d0914 service glance] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:12.204206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09c9a8-9ef6-4c47-ae05-ecb99222cd0a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f0503ae1b24ce6a8adb3cfd7973f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:12.204810 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09c9a8-9ef6-4c47-ae05-ecb99222cd0a service swift] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:12.205088 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09c9a8-9ef6-4c47-ae05-ecb99222cd0a service swift] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:12.205359 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09c9a8-9ef6-4c47-ae05-ecb99222cd0a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:12.211273 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:12.213627 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09c9a8-9ef6-4c47-ae05-ecb99222cd0a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:12.215246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09c9a8-9ef6-4c47-ae05-ecb99222cd0a service swift] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:12.223233 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1317/2652] 10.222.0.59 () {60 vars in 1384 bytes} [Mon Aug 29 23:20:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 23:20:21.964903 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=90 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:22.277404 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:22.894428 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178a1e4-5d0c-4730-ac5b-26d831df63c7 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:22.894713 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178a1e4-5d0c-4730-ac5b-26d831df63c7 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:22.894989 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178a1e4-5d0c-4730-ac5b-26d831df63c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:22.914581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-f178a1e4-5d0c-4730-ac5b-26d831df63c7 None None] MFA Rules not processed for user `79a2748b61b4451f9215a4f8eea5dea2`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:22.915812 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f178a1e4-5d0c-4730-ac5b-26d831df63c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:22.927932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1336/2653] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:23.207901 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09c9a8-9ef6-4c47-ae05-ecb99222cd0a service swift] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:23.209427 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-eb09c9a8-9ef6-4c47-ae05-ecb99222cd0a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:23.283705 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09c9a8-9ef6-4c47-ae05-ecb99222cd0a service swift] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:23.287351 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c751b4a-e3aa-41d8-b4b2-a3d3953f93b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:23.287952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c751b4a-e3aa-41d8-b4b2-a3d3953f93b0 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:23.288260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c751b4a-e3aa-41d8-b4b2-a3d3953f93b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:23.288530 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c751b4a-e3aa-41d8-b4b2-a3d3953f93b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:23.290197 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3c751b4a-e3aa-41d8-b4b2-a3d3953f93b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:23.300250 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c751b4a-e3aa-41d8-b4b2-a3d3953f93b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:23.301643 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c751b4a-e3aa-41d8-b4b2-a3d3953f93b0 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:23.310430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1318/2654] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:20:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:20:23.729324 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178a1e4-5d0c-4730-ac5b-26d831df63c7 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:23.730819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f178a1e4-5d0c-4730-ac5b-26d831df63c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:23.810254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178a1e4-5d0c-4730-ac5b-26d831df63c7 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:23.815289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e7cc9-6784-4fd2-9c71-f986ba45ab04 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:23.815894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e7cc9-6784-4fd2-9c71-f986ba45ab04 service placement] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:23.816200 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e7cc9-6784-4fd2-9c71-f986ba45ab04 service placement] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:23.816473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e7cc9-6784-4fd2-9c71-f986ba45ab04 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:23.818209 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-371e7cc9-6784-4fd2-9c71-f986ba45ab04 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:23.831037 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e7cc9-6784-4fd2-9c71-f986ba45ab04 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:23.832750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e7cc9-6784-4fd2-9c71-f986ba45ab04 service placement] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:23.842614 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1337/2655] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:20:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:20:26.557725 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02255f99-d239-445a-8aa6-2cc5b90be247 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:26.558276 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02255f99-d239-445a-8aa6-2cc5b90be247 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:26.558534 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02255f99-d239-445a-8aa6-2cc5b90be247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:26.578929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-02255f99-d239-445a-8aa6-2cc5b90be247 None None] MFA Rules not processed for user `980157efae8f43b59dc063fda04c0348`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:26.580311 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-02255f99-d239-445a-8aa6-2cc5b90be247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:26.588357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1319/2656] 10.222.0.59 () {58 vars in 1009 bytes} [Mon Aug 29 23:20:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:26.600683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e7cc9-6784-4fd2-9c71-f986ba45ab04 service placement] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:26.612369 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e7cc9-6784-4fd2-9c71-f986ba45ab04 service placement] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:26.617421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55141097-9a58-4bc3-ae4b-9d38ef308585 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f0503ae1b24ce6a8adb3cfd7973f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:26.618023 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55141097-9a58-4bc3-ae4b-9d38ef308585 service swift] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:26.618282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55141097-9a58-4bc3-ae4b-9d38ef308585 service swift] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:26.618549 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55141097-9a58-4bc3-ae4b-9d38ef308585 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:26.626953 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55141097-9a58-4bc3-ae4b-9d38ef308585 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:26.628568 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55141097-9a58-4bc3-ae4b-9d38ef308585 service swift] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:26.637584 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1338/2657] 10.222.0.59 () {60 vars in 1384 bytes} [Mon Aug 29 23:20:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 23:20:31.079247 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02255f99-d239-445a-8aa6-2cc5b90be247 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:31.081040 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-02255f99-d239-445a-8aa6-2cc5b90be247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:31.094171 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02255f99-d239-445a-8aa6-2cc5b90be247 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:31.113213 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1fee8-2ed4-4b37-90b2-8ff1411cf257 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:31.113985 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1fee8-2ed4-4b37-90b2-8ff1411cf257 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:31.114269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1fee8-2ed4-4b37-90b2-8ff1411cf257 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:31.114553 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1fee8-2ed4-4b37-90b2-8ff1411cf257 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:31.116946 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6dc1fee8-2ed4-4b37-90b2-8ff1411cf257 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:31.134566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1fee8-2ed4-4b37-90b2-8ff1411cf257 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:31.136752 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1fee8-2ed4-4b37-90b2-8ff1411cf257 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:31.147886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1320/2658] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:20:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:20:36.633707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=41 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:37.094436 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55141097-9a58-4bc3-ae4b-9d38ef308585 service swift] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:37.095997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-55141097-9a58-4bc3-ae4b-9d38ef308585 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:37.178571 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55141097-9a58-4bc3-ae4b-9d38ef308585 service swift] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:37.184213 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ff6a2-c157-46f9-9cd2-ab12c6aec4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:37.184861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ff6a2-c157-46f9-9cd2-ab12c6aec4c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:37.185183 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ff6a2-c157-46f9-9cd2-ab12c6aec4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:37.185412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ff6a2-c157-46f9-9cd2-ab12c6aec4c8 admin admin] PATH_INFO: `/v3/users/8ecd044ac97747dc8957d62f89a10a6f` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:37.186688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ff6a2-c157-46f9-9cd2-ab12c6aec4c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ecd044ac97747dc8957d62f89a10a6f)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:37.188721 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ff6a2-c157-46f9-9cd2-ab12c6aec4c8 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:37.359580 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-3d7ff6a2-c157-46f9-9cd2-ab12c6aec4c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ecd044ac97747dc8957d62f89a10a6f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:37.360691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-3d7ff6a2-c157-46f9-9cd2-ab12c6aec4c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ecd044ac97747dc8957d62f89a10a6f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:37.373360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-3d7ff6a2-c157-46f9-9cd2-ab12c6aec4c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ecd044ac97747dc8957d62f89a10a6f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:37.453575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-3d7ff6a2-c157-46f9-9cd2-ab12c6aec4c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ecd044ac97747dc8957d62f89a10a6f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:37.454216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-3d7ff6a2-c157-46f9-9cd2-ab12c6aec4c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ecd044ac97747dc8957d62f89a10a6f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:37.480679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1339/2659] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:20:37 2022] DELETE /identity/v3/users/8ecd044ac97747dc8957d62f89a10a6f => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:37.485488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b105d4-a4b4-4120-9c6e-e5158aa5941b None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:37.486438 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b105d4-a4b4-4120-9c6e-e5158aa5941b None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:37.486848 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b105d4-a4b4-4120-9c6e-e5158aa5941b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:37.525870 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-95b105d4-a4b4-4120-9c6e-e5158aa5941b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:37.527079 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-95b105d4-a4b4-4120-9c6e-e5158aa5941b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:37.601297 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1321/2660] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:37.612621 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ff6a2-c157-46f9-9cd2-ab12c6aec4c8 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:37.614216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3d7ff6a2-c157-46f9-9cd2-ab12c6aec4c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:37.683560 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ff6a2-c157-46f9-9cd2-ab12c6aec4c8 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:37.689352 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ca6f2-4943-4960-a9ed-7aa2010833c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:37.690249 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ca6f2-4943-4960-a9ed-7aa2010833c0 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:37.690697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ca6f2-4943-4960-a9ed-7aa2010833c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:37.691181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ca6f2-4943-4960-a9ed-7aa2010833c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:37.706225 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ca6f2-4943-4960-a9ed-7aa2010833c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:37.708865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ca6f2-4943-4960-a9ed-7aa2010833c0 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:37.727222 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1340/2661] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:20:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:20:38.301570 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b105d4-a4b4-4120-9c6e-e5158aa5941b None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:38.303284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-95b105d4-a4b4-4120-9c6e-e5158aa5941b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:38.322245 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b105d4-a4b4-4120-9c6e-e5158aa5941b None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:38.328566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54358810-d66d-41cb-95f0-8e7a8820800b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:38.329127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54358810-d66d-41cb-95f0-8e7a8820800b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:38.329435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54358810-d66d-41cb-95f0-8e7a8820800b admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:38.329662 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54358810-d66d-41cb-95f0-8e7a8820800b admin admin] PATH_INFO: `/v3/projects/55e6d241d1034523bcface43ee8b7b94` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:38.330796 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54358810-d66d-41cb-95f0-8e7a8820800b admin admin] RBAC: Authorizing `identity:delete_project(project_id=55e6d241d1034523bcface43ee8b7b94)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:38.332727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54358810-d66d-41cb-95f0-8e7a8820800b admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:38.457233 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-54358810-d66d-41cb-95f0-8e7a8820800b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55e6d241d1034523bcface43ee8b7b94', 'request_id': 'req-54358810-d66d-41cb-95f0-8e7a8820800b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:38.522401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-54358810-d66d-41cb-95f0-8e7a8820800b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55e6d241d1034523bcface43ee8b7b94', 'request_id': 'req-54358810-d66d-41cb-95f0-8e7a8820800b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:38.529859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-54358810-d66d-41cb-95f0-8e7a8820800b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55e6d241d1034523bcface43ee8b7b94', 'request_id': 'req-54358810-d66d-41cb-95f0-8e7a8820800b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:38.532860 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:38.543133 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-54358810-d66d-41cb-95f0-8e7a8820800b admin admin] The token cache is being invalidate because project 55e6d241d1034523bcface43ee8b7b94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:20:38.543425 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-54358810-d66d-41cb-95f0-8e7a8820800b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:38.547295 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-54358810-d66d-41cb-95f0-8e7a8820800b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:38.547295 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1322/2662] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:20:38 2022] DELETE /identity/v3/projects/55e6d241d1034523bcface43ee8b7b94 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:38.549989 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ca6f2-4943-4960-a9ed-7aa2010833c0 service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:38.561315 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ca6f2-4943-4960-a9ed-7aa2010833c0 service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:38.585238 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:38.631521 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42eada-b388-4347-bea6-d5fabcf459bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:38.632180 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42eada-b388-4347-bea6-d5fabcf459bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:38.632464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42eada-b388-4347-bea6-d5fabcf459bc admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:38.632739 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42eada-b388-4347-bea6-d5fabcf459bc admin admin] PATH_INFO: `/v3/users/24d56fdb8c3a47d7918566aeb379a313` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:38.633900 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42eada-b388-4347-bea6-d5fabcf459bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=24d56fdb8c3a47d7918566aeb379a313)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:38.636419 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42eada-b388-4347-bea6-d5fabcf459bc admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:38.833611 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8f42eada-b388-4347-bea6-d5fabcf459bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24d56fdb8c3a47d7918566aeb379a313'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:38.834304 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8f42eada-b388-4347-bea6-d5fabcf459bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24d56fdb8c3a47d7918566aeb379a313'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:38.842659 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8f42eada-b388-4347-bea6-d5fabcf459bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24d56fdb8c3a47d7918566aeb379a313'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:38.987206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8f42eada-b388-4347-bea6-d5fabcf459bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24d56fdb8c3a47d7918566aeb379a313'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:38.987944 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-8f42eada-b388-4347-bea6-d5fabcf459bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24d56fdb8c3a47d7918566aeb379a313'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:39.007318 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1341/2663] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:20:38 2022] DELETE /identity/v3/users/24d56fdb8c3a47d7918566aeb379a313 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:39.424839 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54358810-d66d-41cb-95f0-8e7a8820800b admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:39.426352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-54358810-d66d-41cb-95f0-8e7a8820800b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:39.440849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54358810-d66d-41cb-95f0-8e7a8820800b admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:39.511597 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260911ca-9659-4f21-aa09-6fc1cd8c0839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:39.512256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260911ca-9659-4f21-aa09-6fc1cd8c0839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:39.512533 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260911ca-9659-4f21-aa09-6fc1cd8c0839 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:39.512799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260911ca-9659-4f21-aa09-6fc1cd8c0839 admin admin] PATH_INFO: `/v3/projects/77f77aaa012b4618970937324297c220` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:39.513916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260911ca-9659-4f21-aa09-6fc1cd8c0839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77f77aaa012b4618970937324297c220)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:39.515783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260911ca-9659-4f21-aa09-6fc1cd8c0839 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:39.622213 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-260911ca-9659-4f21-aa09-6fc1cd8c0839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77f77aaa012b4618970937324297c220', 'request_id': 'req-260911ca-9659-4f21-aa09-6fc1cd8c0839'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:39.706950 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-260911ca-9659-4f21-aa09-6fc1cd8c0839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77f77aaa012b4618970937324297c220', 'request_id': 'req-260911ca-9659-4f21-aa09-6fc1cd8c0839'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:39.712754 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-260911ca-9659-4f21-aa09-6fc1cd8c0839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77f77aaa012b4618970937324297c220', 'request_id': 'req-260911ca-9659-4f21-aa09-6fc1cd8c0839'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:39.723647 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-260911ca-9659-4f21-aa09-6fc1cd8c0839 admin admin] The token cache is being invalidate because project 77f77aaa012b4618970937324297c220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:20:39.723935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-260911ca-9659-4f21-aa09-6fc1cd8c0839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:39.724596 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-260911ca-9659-4f21-aa09-6fc1cd8c0839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:39.726096 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1323/2664] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:20:39 2022] DELETE /identity/v3/projects/77f77aaa012b4618970937324297c220 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:39.731167 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0641f43-77c6-46c6-87a2-d0974f509475 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:39.731449 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0641f43-77c6-46c6-87a2-d0974f509475 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:39.732068 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0641f43-77c6-46c6-87a2-d0974f509475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:39.753387 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-f0641f43-77c6-46c6-87a2-d0974f509475 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:39.754624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f0641f43-77c6-46c6-87a2-d0974f509475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:39.828125 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1342/2665] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:20:39.837171 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d70ea-29d4-403a-96ba-977f7145eef0 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:39.837459 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d70ea-29d4-403a-96ba-977f7145eef0 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:39.837735 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d70ea-29d4-403a-96ba-977f7145eef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:39.858525 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-a65d70ea-29d4-403a-96ba-977f7145eef0 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:39.859774 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a65d70ea-29d4-403a-96ba-977f7145eef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:39.867948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1324/2666] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:39.873083 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0641f43-77c6-46c6-87a2-d0974f509475 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:39.882842 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0641f43-77c6-46c6-87a2-d0974f509475 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:39.887492 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2f2f6-82d3-42cd-9f79-c14a47e25f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:39.888181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2f2f6-82d3-42cd-9f79-c14a47e25f6d admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:39.888523 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2f2f6-82d3-42cd-9f79-c14a47e25f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:39.888855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2f2f6-82d3-42cd-9f79-c14a47e25f6d admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:39.889557 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2f2f6-82d3-42cd-9f79-c14a47e25f6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:20:39.889919 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2f2f6-82d3-42cd-9f79-c14a47e25f6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:39.891564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2f2f6-82d3-42cd-9f79-c14a47e25f6d admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:39.906756 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1343/2667] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:20:39.908107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d70ea-29d4-403a-96ba-977f7145eef0 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:39.919409 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d70ea-29d4-403a-96ba-977f7145eef0 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:39.925231 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4553e4-f831-4f38-a054-ca0879c6c7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:39.925992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4553e4-f831-4f38-a054-ca0879c6c7c5 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:39.926269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4553e4-f831-4f38-a054-ca0879c6c7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:39.926536 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4553e4-f831-4f38-a054-ca0879c6c7c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:39.927169 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4553e4-f831-4f38-a054-ca0879c6c7c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:39.929119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4553e4-f831-4f38-a054-ca0879c6c7c5 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:40.023161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1325/2668] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:20:39 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:20:40.027447 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2f2f6-82d3-42cd-9f79-c14a47e25f6d admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:40.037994 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2f2f6-82d3-42cd-9f79-c14a47e25f6d admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:40.065731 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:40.096847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf89a15-c8cd-4fd0-bd47-3c2a85a002b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:40.097450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf89a15-c8cd-4fd0-bd47-3c2a85a002b2 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:40.097732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf89a15-c8cd-4fd0-bd47-3c2a85a002b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:40.098005 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf89a15-c8cd-4fd0-bd47-3c2a85a002b2 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:40.098625 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf89a15-c8cd-4fd0-bd47-3c2a85a002b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:40.100610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf89a15-c8cd-4fd0-bd47-3c2a85a002b2 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:40.174632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1344/2669] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:20:40 2022] POST /identity/v3/users => generated 286 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:20:40.175830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4553e4-f831-4f38-a054-ca0879c6c7c5 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:40.188037 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4553e4-f831-4f38-a054-ca0879c6c7c5 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:40.192904 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27da87-112d-410d-bdbe-40fbcaa6e3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:40.193502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27da87-112d-410d-bdbe-40fbcaa6e3ff admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:40.193901 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27da87-112d-410d-bdbe-40fbcaa6e3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:40.194154 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27da87-112d-410d-bdbe-40fbcaa6e3ff admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:40.194785 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27da87-112d-410d-bdbe-40fbcaa6e3ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:40.196404 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27da87-112d-410d-bdbe-40fbcaa6e3ff admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:40.213603 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1326/2670] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:20:40.215187 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf89a15-c8cd-4fd0-bd47-3c2a85a002b2 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:40.225107 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf89a15-c8cd-4fd0-bd47-3c2a85a002b2 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:40.232279 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca197c7-e71f-4b16-a0b1-af6cfb73c2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:40.233065 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca197c7-e71f-4b16-a0b1-af6cfb73c2c2 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:40.233347 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca197c7-e71f-4b16-a0b1-af6cfb73c2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:40.233616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca197c7-e71f-4b16-a0b1-af6cfb73c2c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:40.234688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca197c7-e71f-4b16-a0b1-af6cfb73c2c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:40.236796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca197c7-e71f-4b16-a0b1-af6cfb73c2c2 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:40.252984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1345/2671] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:20:40.257048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27da87-112d-410d-bdbe-40fbcaa6e3ff admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:40.267218 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27da87-112d-410d-bdbe-40fbcaa6e3ff admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:40.271887 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395c8d2-4af1-4c9b-9125-f4d5f54d37bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:40.272584 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395c8d2-4af1-4c9b-9125-f4d5f54d37bd admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:40.272862 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395c8d2-4af1-4c9b-9125-f4d5f54d37bd admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:40.273130 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395c8d2-4af1-4c9b-9125-f4d5f54d37bd admin admin] PATH_INFO: `/v3/projects/35567936eb974b0c867e4ca66a697735/users/6c54a94efadc40bba5699b382453d341/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:40.288997 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395c8d2-4af1-4c9b-9125-f4d5f54d37bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=35567936eb974b0c867e4ca66a697735, user_id=6c54a94efadc40bba5699b382453d341, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:40.292507 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395c8d2-4af1-4c9b-9125-f4d5f54d37bd admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:40.391488 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1327/2672] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:20:40 2022] PUT /identity/v3/projects/35567936eb974b0c867e4ca66a697735/users/6c54a94efadc40bba5699b382453d341/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:40.402372 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d49b0c-d5e4-4e85-8249-d50077a0a966 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:40.402655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d49b0c-d5e4-4e85-8249-d50077a0a966 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:40.402927 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d49b0c-d5e4-4e85-8249-d50077a0a966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:40.430928 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-f3d49b0c-d5e4-4e85-8249-d50077a0a966 None None] MFA Rules not processed for user `6c54a94efadc40bba5699b382453d341`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:40.432155 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f3d49b0c-d5e4-4e85-8249-d50077a0a966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:40.494434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1346/2673] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:40.499286 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395c8d2-4af1-4c9b-9125-f4d5f54d37bd admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:40.530569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:40.571074 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395c8d2-4af1-4c9b-9125-f4d5f54d37bd admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:40.576937 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae049a-1618-4bb4-b19e-a181e318c5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:40.577546 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae049a-1618-4bb4-b19e-a181e318c5d7 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:40.577828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae049a-1618-4bb4-b19e-a181e318c5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:40.578102 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae049a-1618-4bb4-b19e-a181e318c5d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:40.578750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae049a-1618-4bb4-b19e-a181e318c5d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:40.580676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae049a-1618-4bb4-b19e-a181e318c5d7 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:40.703791 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1328/2674] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:20:40 2022] POST /identity/v3/projects => generated 373 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:20:40.708149 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d49b0c-d5e4-4e85-8249-d50077a0a966 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:40.720014 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d49b0c-d5e4-4e85-8249-d50077a0a966 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:40.776825 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d477f-9fd0-464d-a25f-901587ddb573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:40.777432 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d477f-9fd0-464d-a25f-901587ddb573 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:40.777713 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d477f-9fd0-464d-a25f-901587ddb573 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:40.778001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d477f-9fd0-464d-a25f-901587ddb573 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:40.778638 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d477f-9fd0-464d-a25f-901587ddb573 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:40.780606 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d477f-9fd0-464d-a25f-901587ddb573 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:40.867587 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1347/2675] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:20:40 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:20:40.871950 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae049a-1618-4bb4-b19e-a181e318c5d7 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:40.882456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae049a-1618-4bb4-b19e-a181e318c5d7 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:40.888204 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642239a-57a7-4b11-a5c3-eb90417c45d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:40.888806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642239a-57a7-4b11-a5c3-eb90417c45d5 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:40.889089 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642239a-57a7-4b11-a5c3-eb90417c45d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:40.889356 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642239a-57a7-4b11-a5c3-eb90417c45d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:40.889991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642239a-57a7-4b11-a5c3-eb90417c45d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:40.891561 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642239a-57a7-4b11-a5c3-eb90417c45d5 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:40.903273 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1329/2676] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:20:40.907679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d477f-9fd0-464d-a25f-901587ddb573 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:40.917826 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d477f-9fd0-464d-a25f-901587ddb573 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:40.922692 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82a480-5ab3-4cc6-8178-c46414269e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:40.923342 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82a480-5ab3-4cc6-8178-c46414269e01 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:40.923616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82a480-5ab3-4cc6-8178-c46414269e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:40.923884 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82a480-5ab3-4cc6-8178-c46414269e01 admin admin] PATH_INFO: `/v3/projects/192592fbf5a648ffb44cf77a252b8875/users/053f5a8e218b4d9a861a1609189db933/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:40.938617 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf82a480-5ab3-4cc6-8178-c46414269e01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=192592fbf5a648ffb44cf77a252b8875, user_id=053f5a8e218b4d9a861a1609189db933, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:40.941515 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf82a480-5ab3-4cc6-8178-c46414269e01 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:41.033700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1348/2677] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:20:40 2022] PUT /identity/v3/projects/192592fbf5a648ffb44cf77a252b8875/users/053f5a8e218b4d9a861a1609189db933/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:41.037535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642239a-57a7-4b11-a5c3-eb90417c45d5 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:41.047644 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642239a-57a7-4b11-a5c3-eb90417c45d5 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:41.104376 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67ec1f-bf5d-44d6-bf4b-fe6cf1e715fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:41.104977 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67ec1f-bf5d-44d6-bf4b-fe6cf1e715fc admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:41.105257 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67ec1f-bf5d-44d6-bf4b-fe6cf1e715fc admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:41.105529 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67ec1f-bf5d-44d6-bf4b-fe6cf1e715fc admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:41.106166 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67ec1f-bf5d-44d6-bf4b-fe6cf1e715fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:41.107745 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67ec1f-bf5d-44d6-bf4b-fe6cf1e715fc admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:41.117284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1330/2678] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:20:41.122068 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82a480-5ab3-4cc6-8178-c46414269e01 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:41.127460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:41.132766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82a480-5ab3-4cc6-8178-c46414269e01 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:41.138031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92c5c5-fb60-48db-a854-ba94d293a405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:41.138681 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92c5c5-fb60-48db-a854-ba94d293a405 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:41.138959 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92c5c5-fb60-48db-a854-ba94d293a405 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:41.139233 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92c5c5-fb60-48db-a854-ba94d293a405 admin admin] PATH_INFO: `/v3/projects/192592fbf5a648ffb44cf77a252b8875/users/053f5a8e218b4d9a861a1609189db933/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:41.141169 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92c5c5-fb60-48db-a854-ba94d293a405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=192592fbf5a648ffb44cf77a252b8875, user_id=053f5a8e218b4d9a861a1609189db933, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:41.143793 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92c5c5-fb60-48db-a854-ba94d293a405 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:41.154692 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1349/2679] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:20:41 2022] PUT /identity/v3/projects/192592fbf5a648ffb44cf77a252b8875/users/053f5a8e218b4d9a861a1609189db933/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:41.165254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6901e-e620-440a-b898-612b4519714c None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:41.165535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6901e-e620-440a-b898-612b4519714c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:41.165805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6901e-e620-440a-b898-612b4519714c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:41.194674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-0af6901e-e620-440a-b898-612b4519714c None None] MFA Rules not processed for user `053f5a8e218b4d9a861a1609189db933`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:41.195867 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-0af6901e-e620-440a-b898-612b4519714c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:41.265374 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1331/2680] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:41.276643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92c5c5-fb60-48db-a854-ba94d293a405 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:41.278370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-db92c5c5-fb60-48db-a854-ba94d293a405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:41.335966 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92c5c5-fb60-48db-a854-ba94d293a405 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:41.339791 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44acda-2c8e-47c3-99e2-7c06591a4971 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:41.340417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44acda-2c8e-47c3-99e2-7c06591a4971 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:41.340692 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44acda-2c8e-47c3-99e2-7c06591a4971 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:41.340956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44acda-2c8e-47c3-99e2-7c06591a4971 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:41.349888 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44acda-2c8e-47c3-99e2-7c06591a4971 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:41.351301 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44acda-2c8e-47c3-99e2-7c06591a4971 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:41.361400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1350/2681] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:20:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:20:42.238418 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6901e-e620-440a-b898-612b4519714c None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:42.257228 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:42.299700 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6901e-e620-440a-b898-612b4519714c None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:42.304224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a5401-6ba9-4675-888a-1fa38a3a1c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:42.304845 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a5401-6ba9-4675-888a-1fa38a3a1c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:42.305136 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a5401-6ba9-4675-888a-1fa38a3a1c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:42.305384 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a5401-6ba9-4675-888a-1fa38a3a1c50 admin admin] PATH_INFO: `/v3/users/6c54a94efadc40bba5699b382453d341` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:42.306621 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a5401-6ba9-4675-888a-1fa38a3a1c50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c54a94efadc40bba5699b382453d341)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:42.308985 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a5401-6ba9-4675-888a-1fa38a3a1c50 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:42.458377 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e65a5401-6ba9-4675-888a-1fa38a3a1c50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c54a94efadc40bba5699b382453d341'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:42.466208 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e65a5401-6ba9-4675-888a-1fa38a3a1c50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c54a94efadc40bba5699b382453d341'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:42.466894 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e65a5401-6ba9-4675-888a-1fa38a3a1c50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c54a94efadc40bba5699b382453d341'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:42.481062 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e65a5401-6ba9-4675-888a-1fa38a3a1c50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c54a94efadc40bba5699b382453d341'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:42.481709 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e65a5401-6ba9-4675-888a-1fa38a3a1c50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c54a94efadc40bba5699b382453d341'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:42.560818 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1332/2682] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:20:42 2022] DELETE /identity/v3/users/6c54a94efadc40bba5699b382453d341 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:42.565254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570b3ed-f6e4-4f51-8b86-f31a6a17203a None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:42.565712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570b3ed-f6e4-4f51-8b86-f31a6a17203a None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:42.565795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570b3ed-f6e4-4f51-8b86-f31a6a17203a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:42.585562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-8570b3ed-f6e4-4f51-8b86-f31a6a17203a None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:42.586771 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8570b3ed-f6e4-4f51-8b86-f31a6a17203a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:42.649781 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1351/2683] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:42.659591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a5401-6ba9-4675-888a-1fa38a3a1c50 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:42.661021 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-e65a5401-6ba9-4675-888a-1fa38a3a1c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:42.718125 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a5401-6ba9-4675-888a-1fa38a3a1c50 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:42.721695 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbc6ee-3833-4f3b-9775-fa9b28b4cf48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:42.722317 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbc6ee-3833-4f3b-9775-fa9b28b4cf48 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:42.722592 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbc6ee-3833-4f3b-9775-fa9b28b4cf48 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:42.722858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbc6ee-3833-4f3b-9775-fa9b28b4cf48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:42.731671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fbc6ee-3833-4f3b-9775-fa9b28b4cf48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:42.733116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fbc6ee-3833-4f3b-9775-fa9b28b4cf48 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:42.742359 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1333/2684] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:20:43.206414 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570b3ed-f6e4-4f51-8b86-f31a6a17203a None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:43.207819 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8570b3ed-f6e4-4f51-8b86-f31a6a17203a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:43.234026 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570b3ed-f6e4-4f51-8b86-f31a6a17203a None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:43.239292 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272386a6-7e81-4755-9425-cae550b286ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:43.239932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272386a6-7e81-4755-9425-cae550b286ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:43.240252 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272386a6-7e81-4755-9425-cae550b286ec admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:43.240599 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272386a6-7e81-4755-9425-cae550b286ec admin admin] PATH_INFO: `/v3/projects/35567936eb974b0c867e4ca66a697735` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:43.241655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272386a6-7e81-4755-9425-cae550b286ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=35567936eb974b0c867e4ca66a697735)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:43.243657 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272386a6-7e81-4755-9425-cae550b286ec admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:43.348653 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-272386a6-7e81-4755-9425-cae550b286ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35567936eb974b0c867e4ca66a697735', 'request_id': 'req-272386a6-7e81-4755-9425-cae550b286ec'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:43.355836 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-272386a6-7e81-4755-9425-cae550b286ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35567936eb974b0c867e4ca66a697735', 'request_id': 'req-272386a6-7e81-4755-9425-cae550b286ec'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:43.361946 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-272386a6-7e81-4755-9425-cae550b286ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35567936eb974b0c867e4ca66a697735', 'request_id': 'req-272386a6-7e81-4755-9425-cae550b286ec'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:43.451997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-272386a6-7e81-4755-9425-cae550b286ec admin admin] The token cache is being invalidate because project 35567936eb974b0c867e4ca66a697735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:20:43.452371 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-272386a6-7e81-4755-9425-cae550b286ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:43.453021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-272386a6-7e81-4755-9425-cae550b286ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:43.454510 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1352/2685] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:20:43 2022] DELETE /identity/v3/projects/35567936eb974b0c867e4ca66a697735 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:43.458576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbc6ee-3833-4f3b-9775-fa9b28b4cf48 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:43.469239 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbc6ee-3833-4f3b-9775-fa9b28b4cf48 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:43.509318 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:43.528018 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db78431c-b45f-44cb-bb2a-1cda745b38de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:43.528682 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78431c-b45f-44cb-bb2a-1cda745b38de admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:43.528961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78431c-b45f-44cb-bb2a-1cda745b38de admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:43.529236 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78431c-b45f-44cb-bb2a-1cda745b38de admin admin] PATH_INFO: `/v3/users/053f5a8e218b4d9a861a1609189db933` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:43.530347 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db78431c-b45f-44cb-bb2a-1cda745b38de admin admin] RBAC: Authorizing `identity:delete_user(user_id=053f5a8e218b4d9a861a1609189db933)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:43.532535 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db78431c-b45f-44cb-bb2a-1cda745b38de admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:43.708643 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-db78431c-b45f-44cb-bb2a-1cda745b38de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '053f5a8e218b4d9a861a1609189db933'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:43.716324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-db78431c-b45f-44cb-bb2a-1cda745b38de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '053f5a8e218b4d9a861a1609189db933'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:43.717076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-db78431c-b45f-44cb-bb2a-1cda745b38de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '053f5a8e218b4d9a861a1609189db933'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:43.730458 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-db78431c-b45f-44cb-bb2a-1cda745b38de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '053f5a8e218b4d9a861a1609189db933'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:43.731458 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-db78431c-b45f-44cb-bb2a-1cda745b38de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '053f5a8e218b4d9a861a1609189db933'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:43.804841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1334/2686] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:20:43 2022] DELETE /identity/v3/users/053f5a8e218b4d9a861a1609189db933 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:44.204032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272386a6-7e81-4755-9425-cae550b286ec admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:44.205607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-272386a6-7e81-4755-9425-cae550b286ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:44.219493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272386a6-7e81-4755-9425-cae550b286ec admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:44.234873 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:44.278817 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927aa5cb-45fe-4889-aa05-e3876f426f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:44.279381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927aa5cb-45fe-4889-aa05-e3876f426f50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:44.279661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927aa5cb-45fe-4889-aa05-e3876f426f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:44.279932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927aa5cb-45fe-4889-aa05-e3876f426f50 admin admin] PATH_INFO: `/v3/projects/192592fbf5a648ffb44cf77a252b8875` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:44.281134 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927aa5cb-45fe-4889-aa05-e3876f426f50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=192592fbf5a648ffb44cf77a252b8875)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:44.283643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927aa5cb-45fe-4889-aa05-e3876f426f50 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:44.397185 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-927aa5cb-45fe-4889-aa05-e3876f426f50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '192592fbf5a648ffb44cf77a252b8875', 'request_id': 'req-927aa5cb-45fe-4889-aa05-e3876f426f50'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:44.403891 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-927aa5cb-45fe-4889-aa05-e3876f426f50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '192592fbf5a648ffb44cf77a252b8875', 'request_id': 'req-927aa5cb-45fe-4889-aa05-e3876f426f50'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:44.409684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-927aa5cb-45fe-4889-aa05-e3876f426f50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '192592fbf5a648ffb44cf77a252b8875', 'request_id': 'req-927aa5cb-45fe-4889-aa05-e3876f426f50'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:44.502380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-927aa5cb-45fe-4889-aa05-e3876f426f50 admin admin] The token cache is being invalidate because project 192592fbf5a648ffb44cf77a252b8875 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:20:44.502664 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-927aa5cb-45fe-4889-aa05-e3876f426f50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:44.503293 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-927aa5cb-45fe-4889-aa05-e3876f426f50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:44.504805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1353/2687] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:20:44 2022] DELETE /identity/v3/projects/192592fbf5a648ffb44cf77a252b8875 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:44.509666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2563e2-2e59-476b-aee2-c152299fd863 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:44.510631 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2563e2-2e59-476b-aee2-c152299fd863 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:44.510717 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2563e2-2e59-476b-aee2-c152299fd863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:44.531108 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-2e2563e2-2e59-476b-aee2-c152299fd863 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:44.532583 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-2e2563e2-2e59-476b-aee2-c152299fd863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:44.591750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1335/2688] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:44.600481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f92ad5-cb4a-4837-83e8-313ac60408aa None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:44.600796 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f92ad5-cb4a-4837-83e8-313ac60408aa None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:44.601075 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f92ad5-cb4a-4837-83e8-313ac60408aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:44.621600 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-07f92ad5-cb4a-4837-83e8-313ac60408aa None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:44.622822 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-07f92ad5-cb4a-4837-83e8-313ac60408aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:44.630711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1354/2689] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:44.640751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2563e2-2e59-476b-aee2-c152299fd863 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:44.650622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2563e2-2e59-476b-aee2-c152299fd863 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:44.656027 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ad214-8376-45ee-b538-5b457df91547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:44.656648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ad214-8376-45ee-b538-5b457df91547 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:44.656922 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ad214-8376-45ee-b538-5b457df91547 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:44.657437 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ad214-8376-45ee-b538-5b457df91547 admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:44.658068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ad214-8376-45ee-b538-5b457df91547 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:20:44.658362 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ad214-8376-45ee-b538-5b457df91547 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:44.659954 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ad214-8376-45ee-b538-5b457df91547 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:44.671329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1336/2690] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:20:44.675662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f92ad5-cb4a-4837-83e8-313ac60408aa None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:44.685075 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f92ad5-cb4a-4837-83e8-313ac60408aa None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:44.689666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8daeb-893b-42d0-b5b3-2a706c2fce5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:44.690256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8daeb-893b-42d0-b5b3-2a706c2fce5a admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:44.695828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8daeb-893b-42d0-b5b3-2a706c2fce5a admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:44.695828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8daeb-893b-42d0-b5b3-2a706c2fce5a admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:44.695828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8daeb-893b-42d0-b5b3-2a706c2fce5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:44.695828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8daeb-893b-42d0-b5b3-2a706c2fce5a admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:44.759755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1355/2691] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:20:44 2022] POST /identity/v3/projects => generated 365 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:20:44.765025 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ad214-8376-45ee-b538-5b457df91547 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:44.775140 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ad214-8376-45ee-b538-5b457df91547 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:44.820952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:44.833962 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce533b-15ee-4d35-8411-451ca61e1dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:44.838575 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce533b-15ee-4d35-8411-451ca61e1dc5 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:44.838575 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce533b-15ee-4d35-8411-451ca61e1dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:44.838575 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce533b-15ee-4d35-8411-451ca61e1dc5 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:44.838575 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce533b-15ee-4d35-8411-451ca61e1dc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:44.838575 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce533b-15ee-4d35-8411-451ca61e1dc5 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:44.941744 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1337/2692] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:20:44 2022] POST /identity/v3/users => generated 283 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:20:44.946010 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8daeb-893b-42d0-b5b3-2a706c2fce5a admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:44.961146 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8daeb-893b-42d0-b5b3-2a706c2fce5a admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:44.971856 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212a005-2b81-45b4-b61e-b02d695a62d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:44.972600 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212a005-2b81-45b4-b61e-b02d695a62d8 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:44.972877 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212a005-2b81-45b4-b61e-b02d695a62d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:44.973282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212a005-2b81-45b4-b61e-b02d695a62d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:44.973933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212a005-2b81-45b4-b61e-b02d695a62d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:44.975674 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212a005-2b81-45b4-b61e-b02d695a62d8 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:44.987805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1356/2693] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:20:44.992050 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce533b-15ee-4d35-8411-451ca61e1dc5 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:45.002921 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce533b-15ee-4d35-8411-451ca61e1dc5 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:45.007742 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae96e781-b363-4931-88e0-86201b2f8631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:45.008395 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96e781-b363-4931-88e0-86201b2f8631 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:45.008675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96e781-b363-4931-88e0-86201b2f8631 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:45.008950 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96e781-b363-4931-88e0-86201b2f8631 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:45.009582 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae96e781-b363-4931-88e0-86201b2f8631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:45.011751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae96e781-b363-4931-88e0-86201b2f8631 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:45.145254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1338/2694] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:20:45.149559 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212a005-2b81-45b4-b61e-b02d695a62d8 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:45.160583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212a005-2b81-45b4-b61e-b02d695a62d8 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:45.165499 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4afa6-39d6-4f3c-9b4f-10d41aab2cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:45.166149 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4afa6-39d6-4f3c-9b4f-10d41aab2cd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:45.166971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4afa6-39d6-4f3c-9b4f-10d41aab2cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:45.167278 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4afa6-39d6-4f3c-9b4f-10d41aab2cd6 admin admin] PATH_INFO: `/v3/projects/6621b4254c9c4a249a5ec735347c0f45/users/6bede63d7f24400db9154b0c6231d70c/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:45.185492 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4afa6-39d6-4f3c-9b4f-10d41aab2cd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6621b4254c9c4a249a5ec735347c0f45, user_id=6bede63d7f24400db9154b0c6231d70c, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:45.185660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4afa6-39d6-4f3c-9b4f-10d41aab2cd6 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:45.256597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1357/2695] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:20:45 2022] PUT /identity/v3/projects/6621b4254c9c4a249a5ec735347c0f45/users/6bede63d7f24400db9154b0c6231d70c/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:45.267450 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b390f03-4554-4011-a31b-93b7e6db0f83 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:45.267726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b390f03-4554-4011-a31b-93b7e6db0f83 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:45.267992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b390f03-4554-4011-a31b-93b7e6db0f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:45.296195 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-3b390f03-4554-4011-a31b-93b7e6db0f83 None None] MFA Rules not processed for user `6bede63d7f24400db9154b0c6231d70c`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:45.297344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3b390f03-4554-4011-a31b-93b7e6db0f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:45.362660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1339/2696] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:45.367264 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4afa6-39d6-4f3c-9b4f-10d41aab2cd6 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:45.413196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:45.430181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4afa6-39d6-4f3c-9b4f-10d41aab2cd6 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:45.435488 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffb02a-d646-49f0-8482-9796efa70bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:45.436089 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffb02a-d646-49f0-8482-9796efa70bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:45.436399 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffb02a-d646-49f0-8482-9796efa70bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:45.436670 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffb02a-d646-49f0-8482-9796efa70bc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:45.437305 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bffb02a-d646-49f0-8482-9796efa70bc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:45.439210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bffb02a-d646-49f0-8482-9796efa70bc6 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:45.547598 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1358/2697] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:20:45 2022] POST /identity/v3/projects => generated 363 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:20:45.551687 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b390f03-4554-4011-a31b-93b7e6db0f83 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:45.563421 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b390f03-4554-4011-a31b-93b7e6db0f83 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:45.619641 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23354681-1e2b-4424-8b64-858efa4cc292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:45.620274 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23354681-1e2b-4424-8b64-858efa4cc292 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:45.620556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23354681-1e2b-4424-8b64-858efa4cc292 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:45.621323 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23354681-1e2b-4424-8b64-858efa4cc292 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:45.622099 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23354681-1e2b-4424-8b64-858efa4cc292 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:45.625109 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23354681-1e2b-4424-8b64-858efa4cc292 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:45.734532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1340/2698] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:20:45 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:20:45.738636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffb02a-d646-49f0-8482-9796efa70bc6 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:45.749681 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffb02a-d646-49f0-8482-9796efa70bc6 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:45.754396 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af3577-2d6e-44f1-b760-1e7110954f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:45.754982 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af3577-2d6e-44f1-b760-1e7110954f7f admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:45.755252 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af3577-2d6e-44f1-b760-1e7110954f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:45.755517 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af3577-2d6e-44f1-b760-1e7110954f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:45.756161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af3577-2d6e-44f1-b760-1e7110954f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:45.757788 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af3577-2d6e-44f1-b760-1e7110954f7f admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:45.768381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1359/2699] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:20:45.772504 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23354681-1e2b-4424-8b64-858efa4cc292 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:45.782737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23354681-1e2b-4424-8b64-858efa4cc292 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:45.787326 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c802626-d41b-489e-9812-eabd276f292d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:45.787963 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c802626-d41b-489e-9812-eabd276f292d admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:45.788275 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c802626-d41b-489e-9812-eabd276f292d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:45.788545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c802626-d41b-489e-9812-eabd276f292d admin admin] PATH_INFO: `/v3/projects/0863905210e647aaa1a4c3b9b04a586e/users/193dd0a88e484387a8a9ae5f427e79a8/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:45.803726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c802626-d41b-489e-9812-eabd276f292d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0863905210e647aaa1a4c3b9b04a586e, user_id=193dd0a88e484387a8a9ae5f427e79a8, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:45.806330 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c802626-d41b-489e-9812-eabd276f292d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:45.884449 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1341/2700] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:20:45 2022] PUT /identity/v3/projects/0863905210e647aaa1a4c3b9b04a586e/users/193dd0a88e484387a8a9ae5f427e79a8/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:45.888438 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af3577-2d6e-44f1-b760-1e7110954f7f admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:45.898487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af3577-2d6e-44f1-b760-1e7110954f7f admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:45.961157 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5e753-b2fb-41f8-bfc1-20d226f5d092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:45.961763 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5e753-b2fb-41f8-bfc1-20d226f5d092 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:45.962033 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5e753-b2fb-41f8-bfc1-20d226f5d092 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:45.962293 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5e753-b2fb-41f8-bfc1-20d226f5d092 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:45.962911 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5e753-b2fb-41f8-bfc1-20d226f5d092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:45.964551 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5e753-b2fb-41f8-bfc1-20d226f5d092 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:45.973942 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1360/2701] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:20:45.978107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c802626-d41b-489e-9812-eabd276f292d admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:45.989129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c802626-d41b-489e-9812-eabd276f292d admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:45.993815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f92020-f2e3-49be-bf53-2d3154acf0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:45.994539 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f92020-f2e3-49be-bf53-2d3154acf0c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:45.994815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f92020-f2e3-49be-bf53-2d3154acf0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:45.995085 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f92020-f2e3-49be-bf53-2d3154acf0c8 admin admin] PATH_INFO: `/v3/projects/0863905210e647aaa1a4c3b9b04a586e/users/193dd0a88e484387a8a9ae5f427e79a8/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:45.997871 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f92020-f2e3-49be-bf53-2d3154acf0c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0863905210e647aaa1a4c3b9b04a586e, user_id=193dd0a88e484387a8a9ae5f427e79a8, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:46.000500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f92020-f2e3-49be-bf53-2d3154acf0c8 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:46.010657 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1342/2702] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:20:45 2022] PUT /identity/v3/projects/0863905210e647aaa1a4c3b9b04a586e/users/193dd0a88e484387a8a9ae5f427e79a8/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:46.021685 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee725a-e0e0-463b-8a5d-e094e8cb4fac None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:46.021974 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee725a-e0e0-463b-8a5d-e094e8cb4fac None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:46.022245 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee725a-e0e0-463b-8a5d-e094e8cb4fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:46.050942 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-e3ee725a-e0e0-463b-8a5d-e094e8cb4fac None None] MFA Rules not processed for user `193dd0a88e484387a8a9ae5f427e79a8`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:46.052162 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-e3ee725a-e0e0-463b-8a5d-e094e8cb4fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:46.125757 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1361/2703] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:20:46.137445 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f92020-f2e3-49be-bf53-2d3154acf0c8 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:46.139176 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f0f92020-f2e3-49be-bf53-2d3154acf0c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:46.151322 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:46.221342 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f92020-f2e3-49be-bf53-2d3154acf0c8 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:46.225061 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae1b1c-2af3-49d6-8da6-7bfa3d5cd830 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:46.225677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae1b1c-2af3-49d6-8da6-7bfa3d5cd830 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:46.225993 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae1b1c-2af3-49d6-8da6-7bfa3d5cd830 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:46.226281 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae1b1c-2af3-49d6-8da6-7bfa3d5cd830 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:46.237889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae1b1c-2af3-49d6-8da6-7bfa3d5cd830 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:46.239417 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae1b1c-2af3-49d6-8da6-7bfa3d5cd830 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:46.250375 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1343/2704] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:20:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:20:46.442836 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee725a-e0e0-463b-8a5d-e094e8cb4fac None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:46.454210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee725a-e0e0-463b-8a5d-e094e8cb4fac None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:46.461697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64992bd9-a1a1-4e10-9f91-5ab3a8e085de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:46.462442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64992bd9-a1a1-4e10-9f91-5ab3a8e085de service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:46.462794 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64992bd9-a1a1-4e10-9f91-5ab3a8e085de service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:46.463097 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64992bd9-a1a1-4e10-9f91-5ab3a8e085de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:46.499198 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:46.521970 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64992bd9-a1a1-4e10-9f91-5ab3a8e085de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:46.523476 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64992bd9-a1a1-4e10-9f91-5ab3a8e085de service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:46.534683 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1362/2705] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:20:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:20:48.370581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae1b1c-2af3-49d6-8da6-7bfa3d5cd830 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:48.373248 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-2dae1b1c-2af3-49d6-8da6-7bfa3d5cd830 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:48.474859 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae1b1c-2af3-49d6-8da6-7bfa3d5cd830 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:48.481310 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e53459-594a-45d2-abb1-50e372e61d7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:48.481927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e53459-594a-45d2-abb1-50e372e61d7b service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:48.482195 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e53459-594a-45d2-abb1-50e372e61d7b service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:48.482461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e53459-594a-45d2-abb1-50e372e61d7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:48.484261 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f2e53459-594a-45d2-abb1-50e372e61d7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:48.498064 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e53459-594a-45d2-abb1-50e372e61d7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:48.499945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e53459-594a-45d2-abb1-50e372e61d7b service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:48.511131 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1344/2706] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:20:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:20:48.813000 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64992bd9-a1a1-4e10-9f91-5ab3a8e085de service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:48.814458 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-64992bd9-a1a1-4e10-9f91-5ab3a8e085de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:48.884387 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64992bd9-a1a1-4e10-9f91-5ab3a8e085de service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:48.889712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4acbbd-1d18-445d-beae-7fc41e304612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:48.890338 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4acbbd-1d18-445d-beae-7fc41e304612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:48.890616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4acbbd-1d18-445d-beae-7fc41e304612 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:48.890888 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4acbbd-1d18-445d-beae-7fc41e304612 admin admin] PATH_INFO: `/v3/users/79b68aa949d24074be074ef82a541bc5` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:48.892037 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4acbbd-1d18-445d-beae-7fc41e304612 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79b68aa949d24074be074ef82a541bc5)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:48.893998 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4acbbd-1d18-445d-beae-7fc41e304612 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:49.048860 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-da4acbbd-1d18-445d-beae-7fc41e304612 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79b68aa949d24074be074ef82a541bc5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:49.049723 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-da4acbbd-1d18-445d-beae-7fc41e304612 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79b68aa949d24074be074ef82a541bc5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:49.057094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-da4acbbd-1d18-445d-beae-7fc41e304612 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79b68aa949d24074be074ef82a541bc5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:49.128160 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-da4acbbd-1d18-445d-beae-7fc41e304612 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79b68aa949d24074be074ef82a541bc5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:49.128814 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-da4acbbd-1d18-445d-beae-7fc41e304612 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79b68aa949d24074be074ef82a541bc5'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:49.147466 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1363/2707] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:20:48 2022] DELETE /identity/v3/users/79b68aa949d24074be074ef82a541bc5 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:49.152016 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6172d-8938-4b94-bc8c-af3205bc6eca None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:49.153138 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6172d-8938-4b94-bc8c-af3205bc6eca None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:49.153477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6172d-8938-4b94-bc8c-af3205bc6eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:49.172264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-d3a6172d-8938-4b94-bc8c-af3205bc6eca None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:49.173529 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d3a6172d-8938-4b94-bc8c-af3205bc6eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:49.231344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1345/2708] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:49.241222 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4acbbd-1d18-445d-beae-7fc41e304612 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:49.242649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-da4acbbd-1d18-445d-beae-7fc41e304612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:49.299161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4acbbd-1d18-445d-beae-7fc41e304612 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:49.302691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7477e9-e1e5-46d9-855e-38f4062f513f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:49.303287 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7477e9-e1e5-46d9-855e-38f4062f513f service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:49.303563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7477e9-e1e5-46d9-855e-38f4062f513f service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:49.303827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7477e9-e1e5-46d9-855e-38f4062f513f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:49.313654 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7477e9-e1e5-46d9-855e-38f4062f513f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:49.315080 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7477e9-e1e5-46d9-855e-38f4062f513f service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:49.319089 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:49.325932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1364/2709] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:20:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:20:49.754118 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6172d-8938-4b94-bc8c-af3205bc6eca None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:49.755542 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d3a6172d-8938-4b94-bc8c-af3205bc6eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:49.769031 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6172d-8938-4b94-bc8c-af3205bc6eca None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:49.773841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b84033-4757-4810-845b-bf7e96ae11a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:49.774750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b84033-4757-4810-845b-bf7e96ae11a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:49.774750 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b84033-4757-4810-845b-bf7e96ae11a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:49.775034 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b84033-4757-4810-845b-bf7e96ae11a3 admin admin] PATH_INFO: `/v3/projects/a34d8e04191a4b28b7fb3e19ed43b1d8` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:49.776100 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b84033-4757-4810-845b-bf7e96ae11a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a34d8e04191a4b28b7fb3e19ed43b1d8)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:49.779013 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b84033-4757-4810-845b-bf7e96ae11a3 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:49.802791 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:49.911460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c7b84033-4757-4810-845b-bf7e96ae11a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a34d8e04191a4b28b7fb3e19ed43b1d8', 'request_id': 'req-c7b84033-4757-4810-845b-bf7e96ae11a3'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:50.005498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c7b84033-4757-4810-845b-bf7e96ae11a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a34d8e04191a4b28b7fb3e19ed43b1d8', 'request_id': 'req-c7b84033-4757-4810-845b-bf7e96ae11a3'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:50.012241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c7b84033-4757-4810-845b-bf7e96ae11a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a34d8e04191a4b28b7fb3e19ed43b1d8', 'request_id': 'req-c7b84033-4757-4810-845b-bf7e96ae11a3'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:50.023256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c7b84033-4757-4810-845b-bf7e96ae11a3 admin admin] The token cache is being invalidate because project a34d8e04191a4b28b7fb3e19ed43b1d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:20:50.023544 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c7b84033-4757-4810-845b-bf7e96ae11a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:50.024206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c7b84033-4757-4810-845b-bf7e96ae11a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:50.025688 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1346/2710] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:20:49 2022] DELETE /identity/v3/projects/a34d8e04191a4b28b7fb3e19ed43b1d8 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:50.031092 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb8984-dc96-4595-b555-e4bf0e16025f None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:50.031535 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb8984-dc96-4595-b555-e4bf0e16025f None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:50.031649 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb8984-dc96-4595-b555-e4bf0e16025f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:50.053240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-aceb8984-dc96-4595-b555-e4bf0e16025f None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:50.054436 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-aceb8984-dc96-4595-b555-e4bf0e16025f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:50.114367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1365/2711] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:50.122101 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ef077-7c8c-4275-b82e-4a0d2e1bf038 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:50.122385 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ef077-7c8c-4275-b82e-4a0d2e1bf038 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:50.122662 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ef077-7c8c-4275-b82e-4a0d2e1bf038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:50.143261 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-ec8ef077-7c8c-4275-b82e-4a0d2e1bf038 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:50.144501 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ec8ef077-7c8c-4275-b82e-4a0d2e1bf038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:50.153179 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1347/2712] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:50.157635 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb8984-dc96-4595-b555-e4bf0e16025f None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:50.167324 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb8984-dc96-4595-b555-e4bf0e16025f None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:50.172997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18db3a-d9ae-4595-b6b6-0a9df9b3ce0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:50.173586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18db3a-d9ae-4595-b6b6-0a9df9b3ce0c admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:50.173862 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18db3a-d9ae-4595-b6b6-0a9df9b3ce0c admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:50.174131 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18db3a-d9ae-4595-b6b6-0a9df9b3ce0c admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:50.174772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18db3a-d9ae-4595-b6b6-0a9df9b3ce0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:20:50.175070 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18db3a-d9ae-4595-b6b6-0a9df9b3ce0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:50.176707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18db3a-d9ae-4595-b6b6-0a9df9b3ce0c admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:50.188665 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1366/2713] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:20:50.193251 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ef077-7c8c-4275-b82e-4a0d2e1bf038 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:50.204075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ef077-7c8c-4275-b82e-4a0d2e1bf038 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:50.208703 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa822acb-825c-4159-b6c0-1eec62da3021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:50.209291 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa822acb-825c-4159-b6c0-1eec62da3021 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:50.209568 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa822acb-825c-4159-b6c0-1eec62da3021 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:50.209831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa822acb-825c-4159-b6c0-1eec62da3021 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:50.210449 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa822acb-825c-4159-b6c0-1eec62da3021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:50.212330 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa822acb-825c-4159-b6c0-1eec62da3021 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:50.322052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1348/2714] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:20:50 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:20:50.326760 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18db3a-d9ae-4595-b6b6-0a9df9b3ce0c admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:50.336885 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18db3a-d9ae-4595-b6b6-0a9df9b3ce0c admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:50.397280 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66519d-abaa-495a-b6ab-9ba2c18db442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:50.397879 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66519d-abaa-495a-b6ab-9ba2c18db442 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:50.398153 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66519d-abaa-495a-b6ab-9ba2c18db442 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:50.398417 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66519d-abaa-495a-b6ab-9ba2c18db442 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:50.399069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66519d-abaa-495a-b6ab-9ba2c18db442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:50.400983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66519d-abaa-495a-b6ab-9ba2c18db442 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:50.503049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1367/2715] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:20:50 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:20:50.507123 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa822acb-825c-4159-b6c0-1eec62da3021 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:50.518474 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa822acb-825c-4159-b6c0-1eec62da3021 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:50.523851 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e514b43-4a83-4f0e-b691-deb82a34a5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:50.524506 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e514b43-4a83-4f0e-b691-deb82a34a5ec admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:50.524787 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e514b43-4a83-4f0e-b691-deb82a34a5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:50.525054 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e514b43-4a83-4f0e-b691-deb82a34a5ec admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:50.525696 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e514b43-4a83-4f0e-b691-deb82a34a5ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:50.527284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e514b43-4a83-4f0e-b691-deb82a34a5ec admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:50.539175 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1349/2716] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:20:50.549309 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66519d-abaa-495a-b6ab-9ba2c18db442 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:50.559743 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66519d-abaa-495a-b6ab-9ba2c18db442 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:50.564707 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eace65-be32-4fce-82b0-252fc73280e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:50.565307 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eace65-be32-4fce-82b0-252fc73280e6 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:50.565585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eace65-be32-4fce-82b0-252fc73280e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:50.565858 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eace65-be32-4fce-82b0-252fc73280e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:50.566498 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eace65-be32-4fce-82b0-252fc73280e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:50.568172 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eace65-be32-4fce-82b0-252fc73280e6 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:50.579396 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1368/2717] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:20:50.583722 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e514b43-4a83-4f0e-b691-deb82a34a5ec admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:50.594580 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e514b43-4a83-4f0e-b691-deb82a34a5ec admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:50.599317 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1518d-b20d-4989-91fa-9997447d2026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:50.599962 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1518d-b20d-4989-91fa-9997447d2026 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:50.600277 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1518d-b20d-4989-91fa-9997447d2026 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:50.603814 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1518d-b20d-4989-91fa-9997447d2026 admin admin] PATH_INFO: `/v3/projects/4f7c1328f2744afdb9241fb7b5d959fd/users/56cb8f5762a64017ae2b74147fb892cf/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:50.615890 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e1518d-b20d-4989-91fa-9997447d2026 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f7c1328f2744afdb9241fb7b5d959fd, user_id=56cb8f5762a64017ae2b74147fb892cf, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:50.618550 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e1518d-b20d-4989-91fa-9997447d2026 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:50.685924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1350/2718] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:20:50 2022] PUT /identity/v3/projects/4f7c1328f2744afdb9241fb7b5d959fd/users/56cb8f5762a64017ae2b74147fb892cf/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:50.696362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239b8d7-9734-42c9-91f5-11fbbb7bab62 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:50.696648 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239b8d7-9734-42c9-91f5-11fbbb7bab62 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:50.696921 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239b8d7-9734-42c9-91f5-11fbbb7bab62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:50.724956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-7239b8d7-9734-42c9-91f5-11fbbb7bab62 None None] MFA Rules not processed for user `56cb8f5762a64017ae2b74147fb892cf`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:50.726269 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-7239b8d7-9734-42c9-91f5-11fbbb7bab62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:50.793295 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:50.809322 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1369/2719] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:50.820032 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1518d-b20d-4989-91fa-9997447d2026 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:50.821913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-30e1518d-b20d-4989-91fa-9997447d2026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:50.890591 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1518d-b20d-4989-91fa-9997447d2026 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:50.895014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30ff57-de5a-4307-bb4e-97a2cdf3c41d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:50.895632 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30ff57-de5a-4307-bb4e-97a2cdf3c41d service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:50.895911 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30ff57-de5a-4307-bb4e-97a2cdf3c41d service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:50.896356 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30ff57-de5a-4307-bb4e-97a2cdf3c41d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:50.906577 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b30ff57-de5a-4307-bb4e-97a2cdf3c41d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:50.909974 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b30ff57-de5a-4307-bb4e-97a2cdf3c41d service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:50.926600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1351/2720] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:20:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:20:54.785181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60bc79-b381-4fc4-b719-d70f7c4341e2 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:54.785744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60bc79-b381-4fc4-b719-d70f7c4341e2 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:54.785744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60bc79-b381-4fc4-b719-d70f7c4341e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:54.805878 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-3f60bc79-b381-4fc4-b719-d70f7c4341e2 None None] MFA Rules not processed for user `56cb8f5762a64017ae2b74147fb892cf`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:54.807087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3f60bc79-b381-4fc4-b719-d70f7c4341e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:54.814165 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1370/2721] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:58.696970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30ff57-de5a-4307-bb4e-97a2cdf3c41d service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:58.698520 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-0b30ff57-de5a-4307-bb4e-97a2cdf3c41d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:58.752672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:20:58.772480 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30ff57-de5a-4307-bb4e-97a2cdf3c41d service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:58.777396 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d2b3e-2e2f-484a-adc0-7befcfe35b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:58.778048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d2b3e-2e2f-484a-adc0-7befcfe35b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:58.778324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d2b3e-2e2f-484a-adc0-7befcfe35b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:58.779170 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d2b3e-2e2f-484a-adc0-7befcfe35b92 admin admin] PATH_INFO: `/v3/users/6bede63d7f24400db9154b0c6231d70c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:58.780962 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1d2b3e-2e2f-484a-adc0-7befcfe35b92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bede63d7f24400db9154b0c6231d70c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:58.783201 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1d2b3e-2e2f-484a-adc0-7befcfe35b92 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:58.946530 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1b1d2b3e-2e2f-484a-adc0-7befcfe35b92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bede63d7f24400db9154b0c6231d70c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:58.954366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1b1d2b3e-2e2f-484a-adc0-7befcfe35b92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bede63d7f24400db9154b0c6231d70c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:58.955055 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1b1d2b3e-2e2f-484a-adc0-7befcfe35b92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bede63d7f24400db9154b0c6231d70c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:58.969147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1b1d2b3e-2e2f-484a-adc0-7befcfe35b92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bede63d7f24400db9154b0c6231d70c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:58.970547 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1b1d2b3e-2e2f-484a-adc0-7befcfe35b92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bede63d7f24400db9154b0c6231d70c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:59.045153 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1352/2722] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:20:58 2022] DELETE /identity/v3/users/6bede63d7f24400db9154b0c6231d70c => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:59.049295 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446bfd0-817e-4125-b289-ec133fc2ce93 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:59.049748 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446bfd0-817e-4125-b289-ec133fc2ce93 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:59.049843 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446bfd0-817e-4125-b289-ec133fc2ce93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:59.070219 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-0446bfd0-817e-4125-b289-ec133fc2ce93 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:20:59.071420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-0446bfd0-817e-4125-b289-ec133fc2ce93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:59.134420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1371/2723] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:20:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:20:59.145169 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d2b3e-2e2f-484a-adc0-7befcfe35b92 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:59.146667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-1b1d2b3e-2e2f-484a-adc0-7befcfe35b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:59.207629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d2b3e-2e2f-484a-adc0-7befcfe35b92 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:59.211381 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1614c0-fa13-48ee-92c3-2868dd026b9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:59.212035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1614c0-fa13-48ee-92c3-2868dd026b9e service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:59.212339 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1614c0-fa13-48ee-92c3-2868dd026b9e service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:59.212607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1614c0-fa13-48ee-92c3-2868dd026b9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:59.222783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1614c0-fa13-48ee-92c3-2868dd026b9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:59.224229 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1614c0-fa13-48ee-92c3-2868dd026b9e service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:59.235111 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1353/2724] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:20:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:20:59.658982 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446bfd0-817e-4125-b289-ec133fc2ce93 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:20:59.660818 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-0446bfd0-817e-4125-b289-ec133fc2ce93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:20:59.689365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446bfd0-817e-4125-b289-ec133fc2ce93 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:20:59.694595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171886b9-09fa-4fe2-bca5-353718a15b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:20:59.695166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171886b9-09fa-4fe2-bca5-353718a15b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:20:59.695443 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171886b9-09fa-4fe2-bca5-353718a15b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:20:59.695716 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171886b9-09fa-4fe2-bca5-353718a15b54 admin admin] PATH_INFO: `/v3/projects/6621b4254c9c4a249a5ec735347c0f45` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:20:59.696878 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171886b9-09fa-4fe2-bca5-353718a15b54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6621b4254c9c4a249a5ec735347c0f45)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:20:59.698834 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171886b9-09fa-4fe2-bca5-353718a15b54 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:20:59.862585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-171886b9-09fa-4fe2-bca5-353718a15b54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6621b4254c9c4a249a5ec735347c0f45', 'request_id': 'req-171886b9-09fa-4fe2-bca5-353718a15b54'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:59.870789 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-171886b9-09fa-4fe2-bca5-353718a15b54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6621b4254c9c4a249a5ec735347c0f45', 'request_id': 'req-171886b9-09fa-4fe2-bca5-353718a15b54'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:59.878957 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-171886b9-09fa-4fe2-bca5-353718a15b54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6621b4254c9c4a249a5ec735347c0f45', 'request_id': 'req-171886b9-09fa-4fe2-bca5-353718a15b54'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:59.991318 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-171886b9-09fa-4fe2-bca5-353718a15b54 admin admin] The token cache is being invalidate because project 6621b4254c9c4a249a5ec735347c0f45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:20:59.991585 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-171886b9-09fa-4fe2-bca5-353718a15b54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:59.992167 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-171886b9-09fa-4fe2-bca5-353718a15b54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:20:59.993757 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1372/2725] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:20:59 2022] DELETE /identity/v3/projects/6621b4254c9c4a249a5ec735347c0f45 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:20:59.997837 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1614c0-fa13-48ee-92c3-2868dd026b9e service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:00.007659 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1614c0-fa13-48ee-92c3-2868dd026b9e service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:00.066150 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31c92b-1c43-467a-adc7-7b0d64e5ba1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:00.066787 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31c92b-1c43-467a-adc7-7b0d64e5ba1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:00.067058 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31c92b-1c43-467a-adc7-7b0d64e5ba1f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:00.067322 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31c92b-1c43-467a-adc7-7b0d64e5ba1f admin admin] PATH_INFO: `/v3/users/193dd0a88e484387a8a9ae5f427e79a8` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:00.068461 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31c92b-1c43-467a-adc7-7b0d64e5ba1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=193dd0a88e484387a8a9ae5f427e79a8)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:00.070373 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31c92b-1c43-467a-adc7-7b0d64e5ba1f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:00.081104 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:00.277609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-cb31c92b-1c43-467a-adc7-7b0d64e5ba1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '193dd0a88e484387a8a9ae5f427e79a8'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:00.285040 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-cb31c92b-1c43-467a-adc7-7b0d64e5ba1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '193dd0a88e484387a8a9ae5f427e79a8'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:00.285743 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-cb31c92b-1c43-467a-adc7-7b0d64e5ba1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '193dd0a88e484387a8a9ae5f427e79a8'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:00.298832 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-cb31c92b-1c43-467a-adc7-7b0d64e5ba1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '193dd0a88e484387a8a9ae5f427e79a8'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:00.299497 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-cb31c92b-1c43-467a-adc7-7b0d64e5ba1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '193dd0a88e484387a8a9ae5f427e79a8'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:00.380295 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1354/2726] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:20:59 2022] DELETE /identity/v3/users/193dd0a88e484387a8a9ae5f427e79a8 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:00.785641 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171886b9-09fa-4fe2-bca5-353718a15b54 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:00.787196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-171886b9-09fa-4fe2-bca5-353718a15b54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:00.801196 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171886b9-09fa-4fe2-bca5-353718a15b54 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:00.857978 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463052f-2302-4fd0-ae82-4dc279e54b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:00.858597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463052f-2302-4fd0-ae82-4dc279e54b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:00.858871 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463052f-2302-4fd0-ae82-4dc279e54b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:00.859139 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463052f-2302-4fd0-ae82-4dc279e54b4f admin admin] PATH_INFO: `/v3/projects/0863905210e647aaa1a4c3b9b04a586e` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:00.860254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1463052f-2302-4fd0-ae82-4dc279e54b4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0863905210e647aaa1a4c3b9b04a586e)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:00.862132 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1463052f-2302-4fd0-ae82-4dc279e54b4f admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:00.872929 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:00.961797 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1463052f-2302-4fd0-ae82-4dc279e54b4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0863905210e647aaa1a4c3b9b04a586e', 'request_id': 'req-1463052f-2302-4fd0-ae82-4dc279e54b4f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:00.969103 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1463052f-2302-4fd0-ae82-4dc279e54b4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0863905210e647aaa1a4c3b9b04a586e', 'request_id': 'req-1463052f-2302-4fd0-ae82-4dc279e54b4f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:00.974881 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1463052f-2302-4fd0-ae82-4dc279e54b4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0863905210e647aaa1a4c3b9b04a586e', 'request_id': 'req-1463052f-2302-4fd0-ae82-4dc279e54b4f'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:01.034720 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1463052f-2302-4fd0-ae82-4dc279e54b4f admin admin] The token cache is being invalidate because project 0863905210e647aaa1a4c3b9b04a586e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:21:01.035011 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1463052f-2302-4fd0-ae82-4dc279e54b4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:01.035645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-1463052f-2302-4fd0-ae82-4dc279e54b4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:01.037217 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1373/2727] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:21:00 2022] DELETE /identity/v3/projects/0863905210e647aaa1a4c3b9b04a586e => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:01.042077 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b9332-438e-48fd-b8e9-8062a127174d None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:01.042553 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b9332-438e-48fd-b8e9-8062a127174d None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:01.042746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b9332-438e-48fd-b8e9-8062a127174d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:01.063618 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-421b9332-438e-48fd-b8e9-8062a127174d None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:01.064892 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-421b9332-438e-48fd-b8e9-8062a127174d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:01.126138 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1355/2728] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:01.133903 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03cd59-5eaf-491a-9e16-e058419d32c3 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:01.134191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03cd59-5eaf-491a-9e16-e058419d32c3 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:01.134463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03cd59-5eaf-491a-9e16-e058419d32c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:01.155643 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-0b03cd59-5eaf-491a-9e16-e058419d32c3 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:01.156850 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-0b03cd59-5eaf-491a-9e16-e058419d32c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:01.164816 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1374/2729] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:01.168967 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b9332-438e-48fd-b8e9-8062a127174d None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:01.179277 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b9332-438e-48fd-b8e9-8062a127174d None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:01.183793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2828ce1-9597-45ed-9b28-fa0f68e1991e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:01.184419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2828ce1-9597-45ed-9b28-fa0f68e1991e admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:01.184691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2828ce1-9597-45ed-9b28-fa0f68e1991e admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:01.184997 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2828ce1-9597-45ed-9b28-fa0f68e1991e admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:01.185633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2828ce1-9597-45ed-9b28-fa0f68e1991e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:21:01.185922 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2828ce1-9597-45ed-9b28-fa0f68e1991e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:01.187473 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2828ce1-9597-45ed-9b28-fa0f68e1991e admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:01.198545 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1356/2730] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:21:01.202876 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03cd59-5eaf-491a-9e16-e058419d32c3 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:01.211995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03cd59-5eaf-491a-9e16-e058419d32c3 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:01.217236 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c800266-49db-41e6-b102-8ac5ac79fb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:01.217818 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c800266-49db-41e6-b102-8ac5ac79fb02 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:01.218093 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c800266-49db-41e6-b102-8ac5ac79fb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:01.218362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c800266-49db-41e6-b102-8ac5ac79fb02 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:01.218992 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c800266-49db-41e6-b102-8ac5ac79fb02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:01.220900 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c800266-49db-41e6-b102-8ac5ac79fb02 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:01.310118 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1375/2731] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:21:01 2022] POST /identity/v3/projects => generated 393 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:21:01.314164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2828ce1-9597-45ed-9b28-fa0f68e1991e admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:01.324211 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2828ce1-9597-45ed-9b28-fa0f68e1991e admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:01.380213 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35b0c3-9d4b-4516-9d85-2e1d0c3d4309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:01.380903 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35b0c3-9d4b-4516-9d85-2e1d0c3d4309 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:01.381182 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35b0c3-9d4b-4516-9d85-2e1d0c3d4309 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:01.381452 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35b0c3-9d4b-4516-9d85-2e1d0c3d4309 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:01.382082 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35b0c3-9d4b-4516-9d85-2e1d0c3d4309 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:01.384003 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35b0c3-9d4b-4516-9d85-2e1d0c3d4309 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:01.460693 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1357/2732] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:21:01 2022] POST /identity/v3/users => generated 297 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:21:01.461757 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c800266-49db-41e6-b102-8ac5ac79fb02 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:01.472616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c800266-49db-41e6-b102-8ac5ac79fb02 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:01.477341 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a8e14-9309-44d8-ac77-ed6a8257375f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:01.477861 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a8e14-9309-44d8-ac77-ed6a8257375f admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:01.478138 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a8e14-9309-44d8-ac77-ed6a8257375f admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:01.478406 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a8e14-9309-44d8-ac77-ed6a8257375f admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:01.483271 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a8e14-9309-44d8-ac77-ed6a8257375f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:01.483271 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a8e14-9309-44d8-ac77-ed6a8257375f admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:01.612705 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1376/2733] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:01.616743 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35b0c3-9d4b-4516-9d85-2e1d0c3d4309 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:01.623153 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:01.626892 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35b0c3-9d4b-4516-9d85-2e1d0c3d4309 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:01.631781 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501ae4e-f905-4023-922a-4beac3e6fa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:01.632444 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501ae4e-f905-4023-922a-4beac3e6fa43 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:01.632725 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501ae4e-f905-4023-922a-4beac3e6fa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:01.632995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501ae4e-f905-4023-922a-4beac3e6fa43 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:01.633638 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501ae4e-f905-4023-922a-4beac3e6fa43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:01.635238 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501ae4e-f905-4023-922a-4beac3e6fa43 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:01.647262 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1358/2734] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:01.651413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a8e14-9309-44d8-ac77-ed6a8257375f admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:01.662313 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a8e14-9309-44d8-ac77-ed6a8257375f admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:01.666966 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f5676-89f2-46b9-a75e-2a314c9a3b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:01.667600 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f5676-89f2-46b9-a75e-2a314c9a3b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:01.667872 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f5676-89f2-46b9-a75e-2a314c9a3b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:01.668158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f5676-89f2-46b9-a75e-2a314c9a3b7f admin admin] PATH_INFO: `/v3/projects/6ecd7f287e40438688a945a839dc88f5/users/c959fd2ca7fb47de83d40659f22c13e7/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:01.682901 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f5676-89f2-46b9-a75e-2a314c9a3b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ecd7f287e40438688a945a839dc88f5, user_id=c959fd2ca7fb47de83d40659f22c13e7, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:01.685767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f5676-89f2-46b9-a75e-2a314c9a3b7f admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:01.759771 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1377/2735] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:21:01 2022] PUT /identity/v3/projects/6ecd7f287e40438688a945a839dc88f5/users/c959fd2ca7fb47de83d40659f22c13e7/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:01.770600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e5884-2b63-4c3f-bcde-3d5d859b145c None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:01.770881 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e5884-2b63-4c3f-bcde-3d5d859b145c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:01.771147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e5884-2b63-4c3f-bcde-3d5d859b145c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:01.799036 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-0e8e5884-2b63-4c3f-bcde-3d5d859b145c None None] MFA Rules not processed for user `c959fd2ca7fb47de83d40659f22c13e7`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:01.800261 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-0e8e5884-2b63-4c3f-bcde-3d5d859b145c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:01.863832 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1359/2736] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:01.868245 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f5676-89f2-46b9-a75e-2a314c9a3b7f admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:01.929244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f5676-89f2-46b9-a75e-2a314c9a3b7f admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:01.933963 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87dd6f-84fc-407c-8f18-6c9cffb82d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:01.934562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87dd6f-84fc-407c-8f18-6c9cffb82d60 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:01.935363 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87dd6f-84fc-407c-8f18-6c9cffb82d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:01.935629 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87dd6f-84fc-407c-8f18-6c9cffb82d60 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:01.936456 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87dd6f-84fc-407c-8f18-6c9cffb82d60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:01.938324 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87dd6f-84fc-407c-8f18-6c9cffb82d60 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:02.017814 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1378/2737] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:21:01 2022] POST /identity/v3/projects => generated 393 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:21:02.021940 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e5884-2b63-4c3f-bcde-3d5d859b145c None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:02.033544 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e5884-2b63-4c3f-bcde-3d5d859b145c None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:02.088724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133c750-84b6-4982-9697-cc75e7503ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:02.089381 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133c750-84b6-4982-9697-cc75e7503ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:02.089592 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133c750-84b6-4982-9697-cc75e7503ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:02.089860 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133c750-84b6-4982-9697-cc75e7503ad5 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:02.090478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133c750-84b6-4982-9697-cc75e7503ad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:02.092968 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133c750-84b6-4982-9697-cc75e7503ad5 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:02.177936 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1360/2738] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:21:02 2022] POST /identity/v3/users => generated 303 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:21:02.181947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87dd6f-84fc-407c-8f18-6c9cffb82d60 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:02.193318 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87dd6f-84fc-407c-8f18-6c9cffb82d60 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:02.198407 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b4f1f-ecdb-43dd-ae01-b2c1e17fc70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:02.199014 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b4f1f-ecdb-43dd-ae01-b2c1e17fc70d admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:02.199294 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b4f1f-ecdb-43dd-ae01-b2c1e17fc70d admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:02.199564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b4f1f-ecdb-43dd-ae01-b2c1e17fc70d admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:02.200260 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b4f1f-ecdb-43dd-ae01-b2c1e17fc70d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:02.202667 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b4f1f-ecdb-43dd-ae01-b2c1e17fc70d admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:02.214450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1379/2739] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:02.218802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133c750-84b6-4982-9697-cc75e7503ad5 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:02.229477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133c750-84b6-4982-9697-cc75e7503ad5 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:02.234288 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd989c-21ad-4248-8b6d-d5a6bd87fc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:02.234948 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd989c-21ad-4248-8b6d-d5a6bd87fc45 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:02.235228 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd989c-21ad-4248-8b6d-d5a6bd87fc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:02.235909 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd989c-21ad-4248-8b6d-d5a6bd87fc45 admin admin] PATH_INFO: `/v3/projects/64ed4056b7d04126ab5155c2d10eb750/users/ee910bc0288b43b787e29619153a50d3/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:02.255302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd989c-21ad-4248-8b6d-d5a6bd87fc45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64ed4056b7d04126ab5155c2d10eb750, user_id=ee910bc0288b43b787e29619153a50d3, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:02.258164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bd989c-21ad-4248-8b6d-d5a6bd87fc45 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:02.334965 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1361/2740] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:21:02 2022] PUT /identity/v3/projects/64ed4056b7d04126ab5155c2d10eb750/users/ee910bc0288b43b787e29619153a50d3/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:02.338937 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b4f1f-ecdb-43dd-ae01-b2c1e17fc70d admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:02.349136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b4f1f-ecdb-43dd-ae01-b2c1e17fc70d admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:02.357426 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:02.404924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d4a3e-2416-42b0-aa93-43b37dad5b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:02.405451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d4a3e-2416-42b0-aa93-43b37dad5b05 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:02.405727 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d4a3e-2416-42b0-aa93-43b37dad5b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:02.405996 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d4a3e-2416-42b0-aa93-43b37dad5b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:02.406621 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d4a3e-2416-42b0-aa93-43b37dad5b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:02.408556 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d4a3e-2416-42b0-aa93-43b37dad5b05 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:02.418328 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1380/2741] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:02.423201 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd989c-21ad-4248-8b6d-d5a6bd87fc45 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:02.433857 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd989c-21ad-4248-8b6d-d5a6bd87fc45 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:02.439143 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baaee98-c495-4a15-84e8-04e904fe28c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:02.439798 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baaee98-c495-4a15-84e8-04e904fe28c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:02.440103 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baaee98-c495-4a15-84e8-04e904fe28c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:02.440407 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baaee98-c495-4a15-84e8-04e904fe28c9 admin admin] PATH_INFO: `/v3/projects/64ed4056b7d04126ab5155c2d10eb750/users/ee910bc0288b43b787e29619153a50d3/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:02.442412 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baaee98-c495-4a15-84e8-04e904fe28c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64ed4056b7d04126ab5155c2d10eb750, user_id=ee910bc0288b43b787e29619153a50d3, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:02.445106 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baaee98-c495-4a15-84e8-04e904fe28c9 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:02.455654 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1362/2742] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:21:02 2022] PUT /identity/v3/projects/64ed4056b7d04126ab5155c2d10eb750/users/ee910bc0288b43b787e29619153a50d3/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:02.467481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b31d13-8516-447f-a218-85992bc13a37 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:02.467920 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b31d13-8516-447f-a218-85992bc13a37 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:02.468391 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b31d13-8516-447f-a218-85992bc13a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:02.501700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-22b31d13-8516-447f-a218-85992bc13a37 None None] MFA Rules not processed for user `ee910bc0288b43b787e29619153a50d3`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:02.502916 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-22b31d13-8516-447f-a218-85992bc13a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:02.573385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1381/2743] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:02 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:02.584081 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baaee98-c495-4a15-84e8-04e904fe28c9 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:02.585768 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-1baaee98-c495-4a15-84e8-04e904fe28c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:02.625990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:02.643328 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baaee98-c495-4a15-84e8-04e904fe28c9 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:02.647032 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128a78c-f1eb-4eee-8013-8679647364f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:02.647633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128a78c-f1eb-4eee-8013-8679647364f5 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:02.647908 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128a78c-f1eb-4eee-8013-8679647364f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:02.648208 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128a78c-f1eb-4eee-8013-8679647364f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:02.661478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128a78c-f1eb-4eee-8013-8679647364f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:02.661478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128a78c-f1eb-4eee-8013-8679647364f5 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:02.671433 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1363/2744] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:02 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:21:02.718954 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b31d13-8516-447f-a218-85992bc13a37 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:02.720441 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-22b31d13-8516-447f-a218-85992bc13a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:02.733381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b31d13-8516-447f-a218-85992bc13a37 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:02.737700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29419f8-c551-43a0-a3a3-169f6ab621b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:02.738563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29419f8-c551-43a0-a3a3-169f6ab621b6 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:02.738563 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29419f8-c551-43a0-a3a3-169f6ab621b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:02.738919 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29419f8-c551-43a0-a3a3-169f6ab621b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:02.804971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29419f8-c551-43a0-a3a3-169f6ab621b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:02.806386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29419f8-c551-43a0-a3a3-169f6ab621b6 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:02.815068 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1382/2745] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:02 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:21:05.208048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128a78c-f1eb-4eee-8013-8679647364f5 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:05.210075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-7128a78c-f1eb-4eee-8013-8679647364f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:05.269837 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128a78c-f1eb-4eee-8013-8679647364f5 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:05.273471 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d0b7f-40ae-4a41-a410-69c43334af61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:05.274364 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d0b7f-40ae-4a41-a410-69c43334af61 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:05.274684 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d0b7f-40ae-4a41-a410-69c43334af61 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:05.275078 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d0b7f-40ae-4a41-a410-69c43334af61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:05.276918 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-fc4d0b7f-40ae-4a41-a410-69c43334af61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:05.341200 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d0b7f-40ae-4a41-a410-69c43334af61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:05.342756 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d0b7f-40ae-4a41-a410-69c43334af61 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:05.352245 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1364/2746] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:21:08.179446 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29419f8-c551-43a0-a3a3-169f6ab621b6 service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:08.213384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=100 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:08.245036 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29419f8-c551-43a0-a3a3-169f6ab621b6 service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:08.250597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bc2af-7e69-4dad-b7e5-3c4660dc9896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:08.250784 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bc2af-7e69-4dad-b7e5-3c4660dc9896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:08.251289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bc2af-7e69-4dad-b7e5-3c4660dc9896 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:08.252178 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bc2af-7e69-4dad-b7e5-3c4660dc9896 admin admin] PATH_INFO: `/v3/users/c959fd2ca7fb47de83d40659f22c13e7` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:08.253416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bc2af-7e69-4dad-b7e5-3c4660dc9896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c959fd2ca7fb47de83d40659f22c13e7)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:08.255408 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bc2af-7e69-4dad-b7e5-3c4660dc9896 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:08.425672 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-316bc2af-7e69-4dad-b7e5-3c4660dc9896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c959fd2ca7fb47de83d40659f22c13e7'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:08.426380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-316bc2af-7e69-4dad-b7e5-3c4660dc9896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c959fd2ca7fb47de83d40659f22c13e7'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:08.433755 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-316bc2af-7e69-4dad-b7e5-3c4660dc9896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c959fd2ca7fb47de83d40659f22c13e7'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:08.535655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-316bc2af-7e69-4dad-b7e5-3c4660dc9896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c959fd2ca7fb47de83d40659f22c13e7'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:08.539158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-316bc2af-7e69-4dad-b7e5-3c4660dc9896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c959fd2ca7fb47de83d40659f22c13e7'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:08.561488 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1383/2747] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:21:08 2022] DELETE /identity/v3/users/c959fd2ca7fb47de83d40659f22c13e7 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:08.565978 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410f652-85f9-47e0-a98f-a7afcba6deee None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:08.566456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410f652-85f9-47e0-a98f-a7afcba6deee None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:08.566638 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410f652-85f9-47e0-a98f-a7afcba6deee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:08.589667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-a410f652-85f9-47e0-a98f-a7afcba6deee None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:08.591036 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a410f652-85f9-47e0-a98f-a7afcba6deee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:08.688044 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1365/2748] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:08.712566 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bc2af-7e69-4dad-b7e5-3c4660dc9896 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:08.714286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-316bc2af-7e69-4dad-b7e5-3c4660dc9896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:08.807216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bc2af-7e69-4dad-b7e5-3c4660dc9896 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:08.813905 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e7a5d-a32b-407a-bfc3-84d1af233a43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:08.814536 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e7a5d-a32b-407a-bfc3-84d1af233a43 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:08.814814 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e7a5d-a32b-407a-bfc3-84d1af233a43 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:08.815083 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e7a5d-a32b-407a-bfc3-84d1af233a43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:08.827537 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e7a5d-a32b-407a-bfc3-84d1af233a43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:08.829450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e7a5d-a32b-407a-bfc3-84d1af233a43 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:08.876230 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1384/2749] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:21:09.666685 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410f652-85f9-47e0-a98f-a7afcba6deee None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:09.668120 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a410f652-85f9-47e0-a98f-a7afcba6deee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:09.686439 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410f652-85f9-47e0-a98f-a7afcba6deee None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:09.691794 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153db349-fea2-4415-beaa-3349495d82fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:09.692656 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153db349-fea2-4415-beaa-3349495d82fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:09.693007 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153db349-fea2-4415-beaa-3349495d82fb admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:09.693344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153db349-fea2-4415-beaa-3349495d82fb admin admin] PATH_INFO: `/v3/projects/6ecd7f287e40438688a945a839dc88f5` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:09.694557 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153db349-fea2-4415-beaa-3349495d82fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ecd7f287e40438688a945a839dc88f5)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:09.696652 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153db349-fea2-4415-beaa-3349495d82fb admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:09.722869 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:09.831495 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-153db349-fea2-4415-beaa-3349495d82fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ecd7f287e40438688a945a839dc88f5', 'request_id': 'req-153db349-fea2-4415-beaa-3349495d82fb'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:09.982317 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-153db349-fea2-4415-beaa-3349495d82fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ecd7f287e40438688a945a839dc88f5', 'request_id': 'req-153db349-fea2-4415-beaa-3349495d82fb'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:09.989004 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-153db349-fea2-4415-beaa-3349495d82fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ecd7f287e40438688a945a839dc88f5', 'request_id': 'req-153db349-fea2-4415-beaa-3349495d82fb'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:10.001205 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-153db349-fea2-4415-beaa-3349495d82fb admin admin] The token cache is being invalidate because project 6ecd7f287e40438688a945a839dc88f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:21:10.001585 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-153db349-fea2-4415-beaa-3349495d82fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:10.002269 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-153db349-fea2-4415-beaa-3349495d82fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:10.003777 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1366/2750] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:21:09 2022] DELETE /identity/v3/projects/6ecd7f287e40438688a945a839dc88f5 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:10.007902 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e7a5d-a32b-407a-bfc3-84d1af233a43 service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:10.027998 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e7a5d-a32b-407a-bfc3-84d1af233a43 service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:10.081151 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:10.090845 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbb45d-6136-4b84-a2d6-d30a3cbda971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:10.092240 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbb45d-6136-4b84-a2d6-d30a3cbda971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:10.092521 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbb45d-6136-4b84-a2d6-d30a3cbda971 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:10.092686 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbb45d-6136-4b84-a2d6-d30a3cbda971 admin admin] PATH_INFO: `/v3/users/ee910bc0288b43b787e29619153a50d3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:10.094039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbb45d-6136-4b84-a2d6-d30a3cbda971 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee910bc0288b43b787e29619153a50d3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:10.096591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbb45d-6136-4b84-a2d6-d30a3cbda971 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:10.291609 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-26bbb45d-6136-4b84-a2d6-d30a3cbda971 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee910bc0288b43b787e29619153a50d3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:10.292277 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-26bbb45d-6136-4b84-a2d6-d30a3cbda971 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee910bc0288b43b787e29619153a50d3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:10.299225 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-26bbb45d-6136-4b84-a2d6-d30a3cbda971 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee910bc0288b43b787e29619153a50d3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:10.364365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-26bbb45d-6136-4b84-a2d6-d30a3cbda971 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee910bc0288b43b787e29619153a50d3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:10.365619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-26bbb45d-6136-4b84-a2d6-d30a3cbda971 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee910bc0288b43b787e29619153a50d3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:10.385969 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1385/2751] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:21:10 2022] DELETE /identity/v3/users/ee910bc0288b43b787e29619153a50d3 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:10.916250 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153db349-fea2-4415-beaa-3349495d82fb admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:10.917737 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-153db349-fea2-4415-beaa-3349495d82fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:10.931727 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153db349-fea2-4415-beaa-3349495d82fb admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:11.006576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4176e3-21db-4b65-b703-63aa29314197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:11.007286 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4176e3-21db-4b65-b703-63aa29314197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:11.007719 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4176e3-21db-4b65-b703-63aa29314197 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:11.008061 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4176e3-21db-4b65-b703-63aa29314197 admin admin] PATH_INFO: `/v3/projects/64ed4056b7d04126ab5155c2d10eb750` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:11.009268 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4176e3-21db-4b65-b703-63aa29314197 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64ed4056b7d04126ab5155c2d10eb750)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:11.011202 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4176e3-21db-4b65-b703-63aa29314197 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:11.127203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1b4176e3-21db-4b65-b703-63aa29314197 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64ed4056b7d04126ab5155c2d10eb750', 'request_id': 'req-1b4176e3-21db-4b65-b703-63aa29314197'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:11.215016 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1b4176e3-21db-4b65-b703-63aa29314197 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64ed4056b7d04126ab5155c2d10eb750', 'request_id': 'req-1b4176e3-21db-4b65-b703-63aa29314197'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:11.221638 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1b4176e3-21db-4b65-b703-63aa29314197 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64ed4056b7d04126ab5155c2d10eb750', 'request_id': 'req-1b4176e3-21db-4b65-b703-63aa29314197'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:11.232951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1b4176e3-21db-4b65-b703-63aa29314197 admin admin] The token cache is being invalidate because project 64ed4056b7d04126ab5155c2d10eb750 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:21:11.233233 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1b4176e3-21db-4b65-b703-63aa29314197 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:11.234746 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1b4176e3-21db-4b65-b703-63aa29314197 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:11.236318 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1367/2752] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:21:10 2022] DELETE /identity/v3/projects/64ed4056b7d04126ab5155c2d10eb750 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:11.241255 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438eae4-4ce0-4032-a3f9-eb0318891dd3 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:11.241714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438eae4-4ce0-4032-a3f9-eb0318891dd3 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:11.241807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438eae4-4ce0-4032-a3f9-eb0318891dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:11.260828 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-f438eae4-4ce0-4032-a3f9-eb0318891dd3 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:11.262042 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f438eae4-4ce0-4032-a3f9-eb0318891dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:11.319771 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1386/2753] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:11.328009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe92d9f-45c8-4c08-81a0-ee5a29fc5c76 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:11.328385 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe92d9f-45c8-4c08-81a0-ee5a29fc5c76 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:11.328730 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe92d9f-45c8-4c08-81a0-ee5a29fc5c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:11.356434 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-efe92d9f-45c8-4c08-81a0-ee5a29fc5c76 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:11.357430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-efe92d9f-45c8-4c08-81a0-ee5a29fc5c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:11.363073 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199bb6c-f90d-4303-b2c4-4bb88084c088 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:11.363352 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199bb6c-f90d-4303-b2c4-4bb88084c088 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:11.363617 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199bb6c-f90d-4303-b2c4-4bb88084c088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:11.366508 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1368/2754] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:11.370331 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe92d9f-45c8-4c08-81a0-ee5a29fc5c76 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:11.380381 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe92d9f-45c8-4c08-81a0-ee5a29fc5c76 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:11.384663 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-a199bb6c-f90d-4303-b2c4-4bb88084c088 None None] MFA Rules not processed for user `980157efae8f43b59dc063fda04c0348`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:11.385196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25436156-e2de-4593-91f4-510c6f3df10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:11.385841 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25436156-e2de-4593-91f4-510c6f3df10c admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:11.385951 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a199bb6c-f90d-4303-b2c4-4bb88084c088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:11.386117 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25436156-e2de-4593-91f4-510c6f3df10c admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:11.386386 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25436156-e2de-4593-91f4-510c6f3df10c admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:11.387026 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25436156-e2de-4593-91f4-510c6f3df10c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:21:11.387327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25436156-e2de-4593-91f4-510c6f3df10c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:11.388984 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25436156-e2de-4593-91f4-510c6f3df10c admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:11.402453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1369/2755] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:21:11.407560 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25436156-e2de-4593-91f4-510c6f3df10c admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:11.417582 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25436156-e2de-4593-91f4-510c6f3df10c admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:11.423232 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ba4b4-59bb-4122-aad7-476f67dd04ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:11.423366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ba4b4-59bb-4122-aad7-476f67dd04ec admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:11.423622 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ba4b4-59bb-4122-aad7-476f67dd04ec admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:11.423888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ba4b4-59bb-4122-aad7-476f67dd04ec admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:11.424584 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ba4b4-59bb-4122-aad7-476f67dd04ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:11.426647 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ba4b4-59bb-4122-aad7-476f67dd04ec admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:11.443616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1387/2756] 10.222.0.59 () {58 vars in 1009 bytes} [Mon Aug 29 23:21:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:21:11.456459 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199bb6c-f90d-4303-b2c4-4bb88084c088 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:11.457872 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a199bb6c-f90d-4303-b2c4-4bb88084c088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:11.502692 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:11.512237 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1370/2757] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:21:11 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:21:11.513262 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ba4b4-59bb-4122-aad7-476f67dd04ec admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:11.533044 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199bb6c-f90d-4303-b2c4-4bb88084c088 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:11.581346 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ba4b4-59bb-4122-aad7-476f67dd04ec admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:11.588458 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c358a-0712-42bc-86b0-6fd97b56b34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:11.588747 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c358a-0712-42bc-86b0-6fd97b56b34f admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:11.589093 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c358a-0712-42bc-86b0-6fd97b56b34f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:11.591031 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c358a-0712-42bc-86b0-6fd97b56b34f admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:11.591031 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c358a-0712-42bc-86b0-6fd97b56b34f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:11.592315 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c358a-0712-42bc-86b0-6fd97b56b34f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:11.599735 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6fcb0-ffc5-4cee-8182-bf994bed6b14 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f0503ae1b24ce6a8adb3cfd7973f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:11.600384 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6fcb0-ffc5-4cee-8182-bf994bed6b14 service swift] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:11.600655 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6fcb0-ffc5-4cee-8182-bf994bed6b14 service swift] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:11.600920 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6fcb0-ffc5-4cee-8182-bf994bed6b14 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:11.611832 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:11.661043 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6fcb0-ffc5-4cee-8182-bf994bed6b14 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:11.662688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6fcb0-ffc5-4cee-8182-bf994bed6b14 service swift] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:11.670749 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1388/2758] 10.222.0.59 () {60 vars in 1384 bytes} [Mon Aug 29 23:21:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 23:21:11.691041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1371/2759] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:21:11 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:21:11.692027 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6fcb0-ffc5-4cee-8182-bf994bed6b14 service swift] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:11.705686 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6fcb0-ffc5-4cee-8182-bf994bed6b14 service swift] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:11.710831 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40613bd-6c87-4994-9dc9-ebeab04472a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:11.711454 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40613bd-6c87-4994-9dc9-ebeab04472a2 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:11.711724 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40613bd-6c87-4994-9dc9-ebeab04472a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:11.712021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40613bd-6c87-4994-9dc9-ebeab04472a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:11.712726 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40613bd-6c87-4994-9dc9-ebeab04472a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:11.714638 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40613bd-6c87-4994-9dc9-ebeab04472a2 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:11.726222 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1389/2760] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:11.731605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c358a-0712-42bc-86b0-6fd97b56b34f admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:11.741816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c358a-0712-42bc-86b0-6fd97b56b34f admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:11.746460 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a016be-e72a-4895-bd30-cd199bbae9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:11.747060 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a016be-e72a-4895-bd30-cd199bbae9aa admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:11.747338 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a016be-e72a-4895-bd30-cd199bbae9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:11.747607 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a016be-e72a-4895-bd30-cd199bbae9aa admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:11.748285 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a016be-e72a-4895-bd30-cd199bbae9aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:11.750470 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a016be-e72a-4895-bd30-cd199bbae9aa admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:11.761185 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1372/2761] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:21:11.766661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40613bd-6c87-4994-9dc9-ebeab04472a2 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:11.776604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40613bd-6c87-4994-9dc9-ebeab04472a2 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:11.782938 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e6a36-2686-48b4-a64d-215a17a6b428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:11.783843 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e6a36-2686-48b4-a64d-215a17a6b428 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:11.784200 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e6a36-2686-48b4-a64d-215a17a6b428 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:11.784463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e6a36-2686-48b4-a64d-215a17a6b428 admin admin] PATH_INFO: `/v3/projects/5add9f5e36b548a4958d1350bad0498c/users/abf7cc465d2a47e0996da5129435930c/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:11.800148 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e6a36-2686-48b4-a64d-215a17a6b428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5add9f5e36b548a4958d1350bad0498c, user_id=abf7cc465d2a47e0996da5129435930c, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:11.802857 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e6a36-2686-48b4-a64d-215a17a6b428 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:11.890413 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1390/2762] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:21:11 2022] PUT /identity/v3/projects/5add9f5e36b548a4958d1350bad0498c/users/abf7cc465d2a47e0996da5129435930c/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:11.902160 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c333b8-b350-41d6-91ee-a73928f552e0 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:11.902554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c333b8-b350-41d6-91ee-a73928f552e0 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:11.902888 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c333b8-b350-41d6-91ee-a73928f552e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:11.933741 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-53c333b8-b350-41d6-91ee-a73928f552e0 None None] MFA Rules not processed for user `abf7cc465d2a47e0996da5129435930c`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:11.934943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-53c333b8-b350-41d6-91ee-a73928f552e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:12.010097 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1373/2763] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:12.011331 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e6a36-2686-48b4-a64d-215a17a6b428 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:12.138636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e6a36-2686-48b4-a64d-215a17a6b428 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:12.146150 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23e90c-ae0d-4c37-bfb8-debd7b4c23c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:12.147052 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23e90c-ae0d-4c37-bfb8-debd7b4c23c9 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:12.156342 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23e90c-ae0d-4c37-bfb8-debd7b4c23c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:12.156342 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23e90c-ae0d-4c37-bfb8-debd7b4c23c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:12.156342 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f23e90c-ae0d-4c37-bfb8-debd7b4c23c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:12.156342 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f23e90c-ae0d-4c37-bfb8-debd7b4c23c9 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:12.268237 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1391/2764] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:21:12 2022] POST /identity/v3/projects => generated 379 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:21:12.271138 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c333b8-b350-41d6-91ee-a73928f552e0 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:12.283111 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c333b8-b350-41d6-91ee-a73928f552e0 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:12.348463 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82aa0c-3361-415d-a2e6-5b36960954d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:12.349128 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82aa0c-3361-415d-a2e6-5b36960954d5 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:12.349509 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82aa0c-3361-415d-a2e6-5b36960954d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:12.350179 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82aa0c-3361-415d-a2e6-5b36960954d5 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:12.350501 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb82aa0c-3361-415d-a2e6-5b36960954d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:12.352568 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb82aa0c-3361-415d-a2e6-5b36960954d5 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:12.471242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1374/2765] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:21:12 2022] POST /identity/v3/users => generated 290 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:21:12.475252 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23e90c-ae0d-4c37-bfb8-debd7b4c23c9 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:12.487370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23e90c-ae0d-4c37-bfb8-debd7b4c23c9 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:12.492757 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ef229-783b-4328-8722-53ee8a84ad12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:12.493377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ef229-783b-4328-8722-53ee8a84ad12 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:12.498821 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ef229-783b-4328-8722-53ee8a84ad12 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:12.498821 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ef229-783b-4328-8722-53ee8a84ad12 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:12.498821 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ef229-783b-4328-8722-53ee8a84ad12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:12.498821 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ef229-783b-4328-8722-53ee8a84ad12 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:12.508691 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1392/2766] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:12.512827 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82aa0c-3361-415d-a2e6-5b36960954d5 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:12.524394 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82aa0c-3361-415d-a2e6-5b36960954d5 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:12.529458 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f434c-38d8-44e5-be7b-b36a83029407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:12.530072 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f434c-38d8-44e5-be7b-b36a83029407 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:12.530350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f434c-38d8-44e5-be7b-b36a83029407 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:12.531523 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f434c-38d8-44e5-be7b-b36a83029407 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:12.532241 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f434c-38d8-44e5-be7b-b36a83029407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:12.534375 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f434c-38d8-44e5-be7b-b36a83029407 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:12.544900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1375/2767] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:12.552640 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ef229-783b-4328-8722-53ee8a84ad12 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:12.563627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ef229-783b-4328-8722-53ee8a84ad12 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:12.570606 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b397833-d9ba-4414-9466-d829fb1a219b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:12.571433 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b397833-d9ba-4414-9466-d829fb1a219b admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:12.571706 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b397833-d9ba-4414-9466-d829fb1a219b admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:12.572549 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b397833-d9ba-4414-9466-d829fb1a219b admin admin] PATH_INFO: `/v3/projects/11c07c11c67c4ae89482f6bb77a5f406/users/b7a7a9e2fbd34babb55c49faca02d6e4/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:12.596402 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b397833-d9ba-4414-9466-d829fb1a219b admin admin] RBAC: Authorizing `identity:create_grant(project_id=11c07c11c67c4ae89482f6bb77a5f406, user_id=b7a7a9e2fbd34babb55c49faca02d6e4, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:12.599494 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b397833-d9ba-4414-9466-d829fb1a219b admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:12.623426 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7c169-f5c6-40f4-84a0-f475a467698d None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:12.623716 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7c169-f5c6-40f4-84a0-f475a467698d None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:12.624119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7c169-f5c6-40f4-84a0-f475a467698d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:12.644094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-18b7c169-f5c6-40f4-84a0-f475a467698d None None] MFA Rules not processed for user `980157efae8f43b59dc063fda04c0348`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:12.645370 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-18b7c169-f5c6-40f4-84a0-f475a467698d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:12.687552 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1393/2768] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:21:12 2022] PUT /identity/v3/projects/11c07c11c67c4ae89482f6bb77a5f406/users/b7a7a9e2fbd34babb55c49faca02d6e4/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:12.698079 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c894de-b999-449b-a077-52566045c735 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:12.698365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c894de-b999-449b-a077-52566045c735 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:12.698630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c894de-b999-449b-a077-52566045c735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:12.700615 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:12.730839 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-76c894de-b999-449b-a077-52566045c735 None None] MFA Rules not processed for user `b7a7a9e2fbd34babb55c49faca02d6e4`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:12.732247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-76c894de-b999-449b-a077-52566045c735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:12.746354 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1376/2769] 10.222.0.59 () {58 vars in 967 bytes} [Mon Aug 29 23:21:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:21:12.760875 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7c169-f5c6-40f4-84a0-f475a467698d None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:12.783045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:12.809871 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1394/2770] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:21:12.814519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c894de-b999-449b-a077-52566045c735 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:12.838877 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7c169-f5c6-40f4-84a0-f475a467698d None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:12.887725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c894de-b999-449b-a077-52566045c735 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:12.894090 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6eb44-4390-450b-98e2-6048f17973f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:12.894698 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6eb44-4390-450b-98e2-6048f17973f5 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:12.895022 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6eb44-4390-450b-98e2-6048f17973f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:12.895241 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6eb44-4390-450b-98e2-6048f17973f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:12.895956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6eb44-4390-450b-98e2-6048f17973f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:12.897965 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6eb44-4390-450b-98e2-6048f17973f5 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:12.913164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379aae0-26d0-4f6d-8370-d86d838a1466 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f0503ae1b24ce6a8adb3cfd7973f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:12.914205 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379aae0-26d0-4f6d-8370-d86d838a1466 service swift] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:12.914343 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379aae0-26d0-4f6d-8370-d86d838a1466 service swift] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:12.914534 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379aae0-26d0-4f6d-8370-d86d838a1466 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:12.973756 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1395/2771] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:21:12 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:21:12.980607 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6eb44-4390-450b-98e2-6048f17973f5 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:12.983094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a379aae0-26d0-4f6d-8370-d86d838a1466 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:12.984768 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a379aae0-26d0-4f6d-8370-d86d838a1466 service swift] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:12.993239 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1377/2772] 10.222.0.59 () {60 vars in 1384 bytes} [Mon Aug 29 23:21:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 23:21:13.056713 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6eb44-4390-450b-98e2-6048f17973f5 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:13.068188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b625715-5934-4e81-bb04-15de352e9e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:13.068188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b625715-5934-4e81-bb04-15de352e9e04 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:13.068188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b625715-5934-4e81-bb04-15de352e9e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:13.068188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b625715-5934-4e81-bb04-15de352e9e04 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:13.068537 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b625715-5934-4e81-bb04-15de352e9e04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:13.068537 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b625715-5934-4e81-bb04-15de352e9e04 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:13.098055 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:13.175492 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1396/2773] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:21:12 2022] POST /identity/v3/users => generated 295 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:21:13.179435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379aae0-26d0-4f6d-8370-d86d838a1466 service swift] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:13.183540 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:13.191124 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379aae0-26d0-4f6d-8370-d86d838a1466 service swift] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:13.195787 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f011d7-21cc-48fd-beaf-bb317a055ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:13.196437 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f011d7-21cc-48fd-beaf-bb317a055ccc admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:13.196722 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f011d7-21cc-48fd-beaf-bb317a055ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:13.196992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f011d7-21cc-48fd-beaf-bb317a055ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:13.197628 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f011d7-21cc-48fd-beaf-bb317a055ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:13.199239 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f011d7-21cc-48fd-beaf-bb317a055ccc admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:13.209912 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1378/2774] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:13.219590 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b625715-5934-4e81-bb04-15de352e9e04 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:13.229624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b625715-5934-4e81-bb04-15de352e9e04 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:13.234451 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe08821-deee-4f9b-9052-3842f6a2d605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:13.235093 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe08821-deee-4f9b-9052-3842f6a2d605 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:13.235374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe08821-deee-4f9b-9052-3842f6a2d605 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:13.235639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe08821-deee-4f9b-9052-3842f6a2d605 admin admin] PATH_INFO: `/v3/projects/ac3716f045d64c9c9543e36c8f81bed3/users/7abeda1177474317a08d9ad4903bf0c3/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:13.250999 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe08821-deee-4f9b-9052-3842f6a2d605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac3716f045d64c9c9543e36c8f81bed3, user_id=7abeda1177474317a08d9ad4903bf0c3, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:13.253652 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe08821-deee-4f9b-9052-3842f6a2d605 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:13.330115 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1397/2775] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:21:13 2022] PUT /identity/v3/projects/ac3716f045d64c9c9543e36c8f81bed3/users/7abeda1177474317a08d9ad4903bf0c3/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:13.334075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f011d7-21cc-48fd-beaf-bb317a055ccc admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:13.344096 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f011d7-21cc-48fd-beaf-bb317a055ccc admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:13.406403 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd10e2-6141-4bc2-af00-0e4683ea1e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:13.407012 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd10e2-6141-4bc2-af00-0e4683ea1e9b admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:13.407289 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd10e2-6141-4bc2-af00-0e4683ea1e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:13.407554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd10e2-6141-4bc2-af00-0e4683ea1e9b admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:13.408253 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd10e2-6141-4bc2-af00-0e4683ea1e9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:13.410070 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd10e2-6141-4bc2-af00-0e4683ea1e9b admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:13.420674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1379/2776] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:21:13.425210 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe08821-deee-4f9b-9052-3842f6a2d605 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:13.434766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe08821-deee-4f9b-9052-3842f6a2d605 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:13.440400 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeae310-946a-43dc-bb3b-bf7c4d4a89db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:13.441053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeae310-946a-43dc-bb3b-bf7c4d4a89db admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:13.441332 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeae310-946a-43dc-bb3b-bf7c4d4a89db admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:13.441598 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeae310-946a-43dc-bb3b-bf7c4d4a89db admin admin] PATH_INFO: `/v3/projects/ac3716f045d64c9c9543e36c8f81bed3/users/7abeda1177474317a08d9ad4903bf0c3/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:13.443656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeae310-946a-43dc-bb3b-bf7c4d4a89db admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac3716f045d64c9c9543e36c8f81bed3, user_id=7abeda1177474317a08d9ad4903bf0c3, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:13.446984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeae310-946a-43dc-bb3b-bf7c4d4a89db admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:13.458059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1398/2777] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:21:13 2022] PUT /identity/v3/projects/ac3716f045d64c9c9543e36c8f81bed3/users/7abeda1177474317a08d9ad4903bf0c3/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:21:13.468496 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22b250-5622-450f-81c9-b50c99c0da9a None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:13.468731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22b250-5622-450f-81c9-b50c99c0da9a None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:13.469002 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22b250-5622-450f-81c9-b50c99c0da9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:13.496861 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-6d22b250-5622-450f-81c9-b50c99c0da9a None None] MFA Rules not processed for user `7abeda1177474317a08d9ad4903bf0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:13.497598 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6d22b250-5622-450f-81c9-b50c99c0da9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:13.569743 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1380/2778] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:13.581263 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeae310-946a-43dc-bb3b-bf7c4d4a89db admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:13.583862 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-cfeae310-946a-43dc-bb3b-bf7c4d4a89db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:13.639630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeae310-946a-43dc-bb3b-bf7c4d4a89db admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:13.643364 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984477c8-0c81-49b3-92d8-a73f42ed9db7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:13.643954 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984477c8-0c81-49b3-92d8-a73f42ed9db7 service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:13.644258 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984477c8-0c81-49b3-92d8-a73f42ed9db7 service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:13.644523 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984477c8-0c81-49b3-92d8-a73f42ed9db7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:13.653434 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984477c8-0c81-49b3-92d8-a73f42ed9db7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:13.654846 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984477c8-0c81-49b3-92d8-a73f42ed9db7 service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:13.664629 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1399/2779] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:21:13.693924 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf5dd5-3ff0-42fb-8360-88e401896a70 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:13.694207 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf5dd5-3ff0-42fb-8360-88e401896a70 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:13.694476 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf5dd5-3ff0-42fb-8360-88e401896a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:13.713925 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-cedf5dd5-3ff0-42fb-8360-88e401896a70 None None] MFA Rules not processed for user `980157efae8f43b59dc063fda04c0348`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:13.715108 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-cedf5dd5-3ff0-42fb-8360-88e401896a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:13.757710 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:13.770873 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1381/2780] 10.222.0.59 () {58 vars in 1009 bytes} [Mon Aug 29 23:21:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:13.782911 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984477c8-0c81-49b3-92d8-a73f42ed9db7 service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:13.786169 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf5dd5-3ff0-42fb-8360-88e401896a70 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:13.787724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-cedf5dd5-3ff0-42fb-8360-88e401896a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:13.800080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf5dd5-3ff0-42fb-8360-88e401896a70 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:13.803759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99705d-a641-47d9-8838-2666e53223b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:13.804410 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99705d-a641-47d9-8838-2666e53223b0 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:13.804692 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99705d-a641-47d9-8838-2666e53223b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:13.804951 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99705d-a641-47d9-8838-2666e53223b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:13.832323 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:13.851405 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984477c8-0c81-49b3-92d8-a73f42ed9db7 service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:13.856660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30b2b9-17be-4cce-affe-795f62925f87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f0503ae1b24ce6a8adb3cfd7973f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:13.857262 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30b2b9-17be-4cce-affe-795f62925f87 service swift] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:13.857571 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30b2b9-17be-4cce-affe-795f62925f87 service swift] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:13.857844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30b2b9-17be-4cce-affe-795f62925f87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:13.863149 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99705d-a641-47d9-8838-2666e53223b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:13.864983 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99705d-a641-47d9-8838-2666e53223b0 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:13.867117 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30b2b9-17be-4cce-affe-795f62925f87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:13.868728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30b2b9-17be-4cce-affe-795f62925f87 service swift] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:13.874034 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1382/2781] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:21:13.877487 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1400/2782] 10.222.0.59 () {60 vars in 1384 bytes} [Mon Aug 29 23:21:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 23:21:14.424918 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252ca07-2357-4f8d-a3d4-f6e0b515d7c0 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:14.425353 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252ca07-2357-4f8d-a3d4-f6e0b515d7c0 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:14.425452 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252ca07-2357-4f8d-a3d4-f6e0b515d7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:14.446168 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-e252ca07-2357-4f8d-a3d4-f6e0b515d7c0 None None] MFA Rules not processed for user `980157efae8f43b59dc063fda04c0348`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:14.447386 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-e252ca07-2357-4f8d-a3d4-f6e0b515d7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:14.455467 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1383/2783] 10.222.0.59 () {58 vars in 967 bytes} [Mon Aug 29 23:21:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:14.467658 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30b2b9-17be-4cce-affe-795f62925f87 service swift] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:14.478885 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30b2b9-17be-4cce-affe-795f62925f87 service swift] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:14.484089 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422bdd2-ab01-46c5-9a51-8c629c34dfee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f0503ae1b24ce6a8adb3cfd7973f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:14.484745 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422bdd2-ab01-46c5-9a51-8c629c34dfee service swift] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:14.485107 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422bdd2-ab01-46c5-9a51-8c629c34dfee service swift] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:14.485505 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422bdd2-ab01-46c5-9a51-8c629c34dfee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:14.493910 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422bdd2-ab01-46c5-9a51-8c629c34dfee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:14.495505 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422bdd2-ab01-46c5-9a51-8c629c34dfee service swift] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:14.503678 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1401/2784] 10.222.0.59 () {60 vars in 1384 bytes} [Mon Aug 29 23:21:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 23:21:17.223505 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252ca07-2357-4f8d-a3d4-f6e0b515d7c0 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:17.225034 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-e252ca07-2357-4f8d-a3d4-f6e0b515d7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:17.293633 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252ca07-2357-4f8d-a3d4-f6e0b515d7c0 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:17.298389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e665f-b896-4a97-a750-b07d229ff2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:17.299022 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e665f-b896-4a97-a750-b07d229ff2f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:17.299357 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e665f-b896-4a97-a750-b07d229ff2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:17.300129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e665f-b896-4a97-a750-b07d229ff2f5 admin admin] PATH_INFO: `/v3/users/56cb8f5762a64017ae2b74147fb892cf` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:17.301427 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e665f-b896-4a97-a750-b07d229ff2f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56cb8f5762a64017ae2b74147fb892cf)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:17.303319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e665f-b896-4a97-a750-b07d229ff2f5 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:17.511309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-594e665f-b896-4a97-a750-b07d229ff2f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56cb8f5762a64017ae2b74147fb892cf'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:17.518196 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-594e665f-b896-4a97-a750-b07d229ff2f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56cb8f5762a64017ae2b74147fb892cf'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:17.518884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-594e665f-b896-4a97-a750-b07d229ff2f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56cb8f5762a64017ae2b74147fb892cf'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:17.530543 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-594e665f-b896-4a97-a750-b07d229ff2f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56cb8f5762a64017ae2b74147fb892cf'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:17.531185 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-594e665f-b896-4a97-a750-b07d229ff2f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56cb8f5762a64017ae2b74147fb892cf'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:17.618064 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1384/2785] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:21:17 2022] DELETE /identity/v3/users/56cb8f5762a64017ae2b74147fb892cf => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:17.622948 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718bb16-c645-48aa-8d53-67103f31d60d None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:17.623442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718bb16-c645-48aa-8d53-67103f31d60d None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:17.623532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718bb16-c645-48aa-8d53-67103f31d60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:17.644056 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-6718bb16-c645-48aa-8d53-67103f31d60d None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:17.645306 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6718bb16-c645-48aa-8d53-67103f31d60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:17.706984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1402/2786] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:17.717235 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e665f-b896-4a97-a750-b07d229ff2f5 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:17.718901 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-594e665f-b896-4a97-a750-b07d229ff2f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:17.725257 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:17.781103 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e665f-b896-4a97-a750-b07d229ff2f5 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:17.786124 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfb27b-af77-4015-b977-f0b806f4a380 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:17.787034 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfb27b-af77-4015-b977-f0b806f4a380 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:17.787939 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfb27b-af77-4015-b977-f0b806f4a380 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:17.788399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfb27b-af77-4015-b977-f0b806f4a380 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:17.798092 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dfb27b-af77-4015-b977-f0b806f4a380 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:17.799780 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dfb27b-af77-4015-b977-f0b806f4a380 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:17.809788 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1385/2787] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:21:17.856523 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718bb16-c645-48aa-8d53-67103f31d60d None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:17.858139 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6718bb16-c645-48aa-8d53-67103f31d60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:17.872486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718bb16-c645-48aa-8d53-67103f31d60d None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:17.877503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffbc0b-e427-48af-93d5-f5e65239c76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:17.878539 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffbc0b-e427-48af-93d5-f5e65239c76f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:17.878702 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffbc0b-e427-48af-93d5-f5e65239c76f admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:17.879000 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffbc0b-e427-48af-93d5-f5e65239c76f admin admin] PATH_INFO: `/v3/users/abf7cc465d2a47e0996da5129435930c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:17.880595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ffbc0b-e427-48af-93d5-f5e65239c76f admin admin] RBAC: Authorizing `identity:delete_user(user_id=abf7cc465d2a47e0996da5129435930c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:17.883234 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ffbc0b-e427-48af-93d5-f5e65239c76f admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:18.076234 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-63ffbc0b-e427-48af-93d5-f5e65239c76f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abf7cc465d2a47e0996da5129435930c'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:18.076918 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-63ffbc0b-e427-48af-93d5-f5e65239c76f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abf7cc465d2a47e0996da5129435930c'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:18.085322 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-63ffbc0b-e427-48af-93d5-f5e65239c76f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abf7cc465d2a47e0996da5129435930c'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:18.091989 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfb27b-af77-4015-b977-f0b806f4a380 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:18.094048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-d6dfb27b-af77-4015-b977-f0b806f4a380 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:18.110938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfb27b-af77-4015-b977-f0b806f4a380 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:18.115759 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40091368-6499-4268-b1d8-fa2d9f72a46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:18.116926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40091368-6499-4268-b1d8-fa2d9f72a46d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:18.116926 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40091368-6499-4268-b1d8-fa2d9f72a46d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:18.117113 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40091368-6499-4268-b1d8-fa2d9f72a46d admin admin] PATH_INFO: `/v3/projects/4f7c1328f2744afdb9241fb7b5d959fd` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:18.118119 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40091368-6499-4268-b1d8-fa2d9f72a46d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f7c1328f2744afdb9241fb7b5d959fd)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:18.119999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40091368-6499-4268-b1d8-fa2d9f72a46d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:18.164145 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-63ffbc0b-e427-48af-93d5-f5e65239c76f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abf7cc465d2a47e0996da5129435930c'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:18.164913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-63ffbc0b-e427-48af-93d5-f5e65239c76f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abf7cc465d2a47e0996da5129435930c'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:18.183783 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1403/2788] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:21:17 2022] DELETE /identity/v3/users/abf7cc465d2a47e0996da5129435930c => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:18.188239 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa2caf-7adf-49e2-bf51-5d1225aaacf4 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:18.188526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa2caf-7adf-49e2-bf51-5d1225aaacf4 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:18.188800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa2caf-7adf-49e2-bf51-5d1225aaacf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:18.210284 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-a0aa2caf-7adf-49e2-bf51-5d1225aaacf4 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:18.211515 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a0aa2caf-7adf-49e2-bf51-5d1225aaacf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:18.229962 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-40091368-6499-4268-b1d8-fa2d9f72a46d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f7c1328f2744afdb9241fb7b5d959fd', 'request_id': 'req-40091368-6499-4268-b1d8-fa2d9f72a46d'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:18.244410 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:18.276475 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1404/2789] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:21:18.286907 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa2caf-7adf-49e2-bf51-5d1225aaacf4 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:18.288362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-a0aa2caf-7adf-49e2-bf51-5d1225aaacf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:18.295501 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-40091368-6499-4268-b1d8-fa2d9f72a46d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f7c1328f2744afdb9241fb7b5d959fd', 'request_id': 'req-40091368-6499-4268-b1d8-fa2d9f72a46d'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:18.302267 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-40091368-6499-4268-b1d8-fa2d9f72a46d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f7c1328f2744afdb9241fb7b5d959fd', 'request_id': 'req-40091368-6499-4268-b1d8-fa2d9f72a46d'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:18.314371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-40091368-6499-4268-b1d8-fa2d9f72a46d admin admin] The token cache is being invalidate because project 4f7c1328f2744afdb9241fb7b5d959fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:21:18.314664 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-40091368-6499-4268-b1d8-fa2d9f72a46d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:18.315844 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-40091368-6499-4268-b1d8-fa2d9f72a46d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:18.318026 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1386/2790] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:21:18 2022] DELETE /identity/v3/projects/4f7c1328f2744afdb9241fb7b5d959fd => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:21:18.322629 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef16f3-1f7e-4ee2-813c-e2632259065e None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:18.322914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef16f3-1f7e-4ee2-813c-e2632259065e None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:18.323184 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef16f3-1f7e-4ee2-813c-e2632259065e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:18.348846 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-6aef16f3-1f7e-4ee2-813c-e2632259065e None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:18.350321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6aef16f3-1f7e-4ee2-813c-e2632259065e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:18.358770 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa2caf-7adf-49e2-bf51-5d1225aaacf4 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:18.418616 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04408854-5489-46cf-bee6-94395b226280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:18.419288 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04408854-5489-46cf-bee6-94395b226280 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:18.426181 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1387/2791] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:21:18.426291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04408854-5489-46cf-bee6-94395b226280 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:18.426291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04408854-5489-46cf-bee6-94395b226280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:18.426291 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-04408854-5489-46cf-bee6-94395b226280 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:18.427548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe00d77-05be-4960-8f08-6263de966ab2 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:18.427833 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe00d77-05be-4960-8f08-6263de966ab2 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:18.428101 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe00d77-05be-4960-8f08-6263de966ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:18.436311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04408854-5489-46cf-bee6-94395b226280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:18.437808 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04408854-5489-46cf-bee6-94395b226280 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:18.448571 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-5fe00d77-05be-4960-8f08-6263de966ab2 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:18.449773 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-5fe00d77-05be-4960-8f08-6263de966ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:18.453267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1405/2792] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:21:18.458076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1388/2793] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:21:18.462792 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04408854-5489-46cf-bee6-94395b226280 service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:18.473259 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04408854-5489-46cf-bee6-94395b226280 service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:18.480802 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5eda4-93ca-4660-a7ba-ac50a2e5f314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:18.481855 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5eda4-93ca-4660-a7ba-ac50a2e5f314 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:18.482363 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5eda4-93ca-4660-a7ba-ac50a2e5f314 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:18.482859 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5eda4-93ca-4660-a7ba-ac50a2e5f314 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:18.484083 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5eda4-93ca-4660-a7ba-ac50a2e5f314 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:21:18.484671 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5eda4-93ca-4660-a7ba-ac50a2e5f314 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:18.488242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5eda4-93ca-4660-a7ba-ac50a2e5f314 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:18.504813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1406/2794] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:21:18.508999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe00d77-05be-4960-8f08-6263de966ab2 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:18.519576 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe00d77-05be-4960-8f08-6263de966ab2 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:18.527406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32197b9b-cf5d-4a8d-944b-363dc10fa146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:18.528009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32197b9b-cf5d-4a8d-944b-363dc10fa146 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:18.528318 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32197b9b-cf5d-4a8d-944b-363dc10fa146 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:18.528582 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32197b9b-cf5d-4a8d-944b-363dc10fa146 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:18.529206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32197b9b-cf5d-4a8d-944b-363dc10fa146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:18.531116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32197b9b-cf5d-4a8d-944b-363dc10fa146 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:18.605670 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:18.612567 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1389/2795] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:21:18 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:21:18.617103 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5eda4-93ca-4660-a7ba-ac50a2e5f314 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:18.634295 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5eda4-93ca-4660-a7ba-ac50a2e5f314 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:18.685638 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e145c3-48ae-4ef6-8c51-f9c913ed497e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:18.686302 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e145c3-48ae-4ef6-8c51-f9c913ed497e admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:18.686578 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e145c3-48ae-4ef6-8c51-f9c913ed497e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:18.686868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e145c3-48ae-4ef6-8c51-f9c913ed497e admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:18.687493 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e145c3-48ae-4ef6-8c51-f9c913ed497e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:18.689513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e145c3-48ae-4ef6-8c51-f9c913ed497e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:18.770194 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1407/2796] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:21:18 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:21:18.771387 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32197b9b-cf5d-4a8d-944b-363dc10fa146 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:18.782031 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32197b9b-cf5d-4a8d-944b-363dc10fa146 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:18.787414 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf568b-f109-4955-8d8c-ff115cf1c036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:18.788021 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf568b-f109-4955-8d8c-ff115cf1c036 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:18.788322 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf568b-f109-4955-8d8c-ff115cf1c036 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:18.788589 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf568b-f109-4955-8d8c-ff115cf1c036 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:18.789214 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf568b-f109-4955-8d8c-ff115cf1c036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:18.790922 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf568b-f109-4955-8d8c-ff115cf1c036 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:18.802694 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1390/2797] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:18.806644 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e145c3-48ae-4ef6-8c51-f9c913ed497e admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:18.817129 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e145c3-48ae-4ef6-8c51-f9c913ed497e admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:18.821900 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754807b2-3e46-4276-8fea-f73e4f7923b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:18.822495 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754807b2-3e46-4276-8fea-f73e4f7923b7 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:18.822769 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754807b2-3e46-4276-8fea-f73e4f7923b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:18.823037 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754807b2-3e46-4276-8fea-f73e4f7923b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:18.823700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754807b2-3e46-4276-8fea-f73e4f7923b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:18.825336 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754807b2-3e46-4276-8fea-f73e4f7923b7 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:18.830765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:18.836259 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1408/2798] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:18.840653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf568b-f109-4955-8d8c-ff115cf1c036 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:18.852264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf568b-f109-4955-8d8c-ff115cf1c036 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:18.856994 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3fa087-ebb2-4539-9044-c5916fdde694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:18.857636 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fa087-ebb2-4539-9044-c5916fdde694 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:18.857913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fa087-ebb2-4539-9044-c5916fdde694 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:18.860642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fa087-ebb2-4539-9044-c5916fdde694 admin admin] PATH_INFO: `/v3/projects/fc6b1df0e5104fd6abb5dcf5497d2cb1/users/d3acbdab00c64d84871545062a5b5311/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:18.873411 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3fa087-ebb2-4539-9044-c5916fdde694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc6b1df0e5104fd6abb5dcf5497d2cb1, user_id=d3acbdab00c64d84871545062a5b5311, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:18.876126 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3fa087-ebb2-4539-9044-c5916fdde694 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:18.889665 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754807b2-3e46-4276-8fea-f73e4f7923b7 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:18.889665 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-754807b2-3e46-4276-8fea-f73e4f7923b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:18.901433 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754807b2-3e46-4276-8fea-f73e4f7923b7 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:18.906139 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659e8db-b9e3-4f05-9efb-ec37972a9dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:18.906772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659e8db-b9e3-4f05-9efb-ec37972a9dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:18.907045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659e8db-b9e3-4f05-9efb-ec37972a9dda admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:18.907331 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659e8db-b9e3-4f05-9efb-ec37972a9dda admin admin] PATH_INFO: `/v3/projects/5add9f5e36b548a4958d1350bad0498c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:18.908711 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659e8db-b9e3-4f05-9efb-ec37972a9dda admin admin] RBAC: Authorizing `identity:delete_project(project_id=5add9f5e36b548a4958d1350bad0498c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:18.911197 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659e8db-b9e3-4f05-9efb-ec37972a9dda admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:18.971987 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1391/2799] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:21:18 2022] PUT /identity/v3/projects/fc6b1df0e5104fd6abb5dcf5497d2cb1/users/d3acbdab00c64d84871545062a5b5311/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:18.977586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7be0c9-29ab-4827-af66-95ac9283d716 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:18.977864 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7be0c9-29ab-4827-af66-95ac9283d716 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:18.978147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7be0c9-29ab-4827-af66-95ac9283d716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:18.996648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-ef7be0c9-29ab-4827-af66-95ac9283d716 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:18.997830 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ef7be0c9-29ab-4827-af66-95ac9283d716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:19.043340 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b659e8db-b9e3-4f05-9efb-ec37972a9dda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5add9f5e36b548a4958d1350bad0498c', 'request_id': 'req-b659e8db-b9e3-4f05-9efb-ec37972a9dda'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:19.049652 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b659e8db-b9e3-4f05-9efb-ec37972a9dda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5add9f5e36b548a4958d1350bad0498c', 'request_id': 'req-b659e8db-b9e3-4f05-9efb-ec37972a9dda'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:19.058849 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b659e8db-b9e3-4f05-9efb-ec37972a9dda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5add9f5e36b548a4958d1350bad0498c', 'request_id': 'req-b659e8db-b9e3-4f05-9efb-ec37972a9dda'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:19.060431 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1392/2800] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:21:19.070123 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7be0c9-29ab-4827-af66-95ac9283d716 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:19.071481 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ef7be0c9-29ab-4827-af66-95ac9283d716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:19.129933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7be0c9-29ab-4827-af66-95ac9283d716 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:19.132072 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b659e8db-b9e3-4f05-9efb-ec37972a9dda admin admin] The token cache is being invalidate because project 5add9f5e36b548a4958d1350bad0498c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:21:19.132397 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b659e8db-b9e3-4f05-9efb-ec37972a9dda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:19.133036 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b659e8db-b9e3-4f05-9efb-ec37972a9dda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:19.134583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1409/2801] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:21:18 2022] DELETE /identity/v3/projects/5add9f5e36b548a4958d1350bad0498c => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:19.138389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659e8db-b9e3-4f05-9efb-ec37972a9dda admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:19.139762 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b659e8db-b9e3-4f05-9efb-ec37972a9dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:19.185150 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628d974-7aad-44a4-9213-bcae94ca6fa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:19.185861 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628d974-7aad-44a4-9213-bcae94ca6fa9 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:19.186145 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628d974-7aad-44a4-9213-bcae94ca6fa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:19.186505 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628d974-7aad-44a4-9213-bcae94ca6fa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:19.188344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3628d974-7aad-44a4-9213-bcae94ca6fa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:19.206830 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659e8db-b9e3-4f05-9efb-ec37972a9dda admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:19.211598 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140729e-bb41-4b4b-950a-47d0baca1ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:19.212254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140729e-bb41-4b4b-950a-47d0baca1ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:19.215440 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140729e-bb41-4b4b-950a-47d0baca1ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:19.215440 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140729e-bb41-4b4b-950a-47d0baca1ea9 admin admin] PATH_INFO: `/v3/users/b7a7a9e2fbd34babb55c49faca02d6e4` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:19.215440 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0140729e-bb41-4b4b-950a-47d0baca1ea9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7a7a9e2fbd34babb55c49faca02d6e4)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:19.215840 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0140729e-bb41-4b4b-950a-47d0baca1ea9 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:19.223910 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:19.252599 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628d974-7aad-44a4-9213-bcae94ca6fa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:19.255102 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628d974-7aad-44a4-9213-bcae94ca6fa9 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:19.264682 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1393/2802] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:21:19.445005 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0140729e-bb41-4b4b-950a-47d0baca1ea9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7a7a9e2fbd34babb55c49faca02d6e4'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:19.445843 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0140729e-bb41-4b4b-950a-47d0baca1ea9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7a7a9e2fbd34babb55c49faca02d6e4'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:19.455477 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0140729e-bb41-4b4b-950a-47d0baca1ea9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7a7a9e2fbd34babb55c49faca02d6e4'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:19.547404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0140729e-bb41-4b4b-950a-47d0baca1ea9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7a7a9e2fbd34babb55c49faca02d6e4'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:19.548129 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-0140729e-bb41-4b4b-950a-47d0baca1ea9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7a7a9e2fbd34babb55c49faca02d6e4'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:19.571105 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1410/2803] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:21:19 2022] DELETE /identity/v3/users/b7a7a9e2fbd34babb55c49faca02d6e4 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:21:19.986141 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628d974-7aad-44a4-9213-bcae94ca6fa9 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:19.997036 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628d974-7aad-44a4-9213-bcae94ca6fa9 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:20.064039 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0a72d-44b8-43e9-98e2-8236d1c7939a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:20.064698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0a72d-44b8-43e9-98e2-8236d1c7939a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:20.065072 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0a72d-44b8-43e9-98e2-8236d1c7939a admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:20.065343 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0a72d-44b8-43e9-98e2-8236d1c7939a admin admin] PATH_INFO: `/v3/projects/11c07c11c67c4ae89482f6bb77a5f406` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:20.066781 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0a72d-44b8-43e9-98e2-8236d1c7939a admin admin] RBAC: Authorizing `identity:delete_project(project_id=11c07c11c67c4ae89482f6bb77a5f406)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:20.068813 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0a72d-44b8-43e9-98e2-8236d1c7939a admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:20.186107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2da0a72d-44b8-43e9-98e2-8236d1c7939a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11c07c11c67c4ae89482f6bb77a5f406', 'request_id': 'req-2da0a72d-44b8-43e9-98e2-8236d1c7939a'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:20.257722 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2da0a72d-44b8-43e9-98e2-8236d1c7939a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11c07c11c67c4ae89482f6bb77a5f406', 'request_id': 'req-2da0a72d-44b8-43e9-98e2-8236d1c7939a'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:20.268758 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2da0a72d-44b8-43e9-98e2-8236d1c7939a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11c07c11c67c4ae89482f6bb77a5f406', 'request_id': 'req-2da0a72d-44b8-43e9-98e2-8236d1c7939a'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:20.277962 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2da0a72d-44b8-43e9-98e2-8236d1c7939a admin admin] The token cache is being invalidate because project 11c07c11c67c4ae89482f6bb77a5f406 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:21:20.278246 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2da0a72d-44b8-43e9-98e2-8236d1c7939a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:20.278917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-2da0a72d-44b8-43e9-98e2-8236d1c7939a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:20.280473 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1394/2804] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:21:19 2022] DELETE /identity/v3/projects/11c07c11c67c4ae89482f6bb77a5f406 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:20.285314 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140729e-bb41-4b4b-950a-47d0baca1ea9 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:20.286919 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-0140729e-bb41-4b4b-950a-47d0baca1ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:20.295089 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:20.301157 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140729e-bb41-4b4b-950a-47d0baca1ea9 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:20.358348 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3356c-0da8-41c6-af0d-c19460e39fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:20.358971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3356c-0da8-41c6-af0d-c19460e39fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:20.359246 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3356c-0da8-41c6-af0d-c19460e39fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:20.359514 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3356c-0da8-41c6-af0d-c19460e39fe9 admin admin] PATH_INFO: `/v3/users/7abeda1177474317a08d9ad4903bf0c3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:20.360708 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc3356c-0da8-41c6-af0d-c19460e39fe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7abeda1177474317a08d9ad4903bf0c3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:20.362602 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc3356c-0da8-41c6-af0d-c19460e39fe9 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:20.546386 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6dc3356c-0da8-41c6-af0d-c19460e39fe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7abeda1177474317a08d9ad4903bf0c3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:20.547109 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6dc3356c-0da8-41c6-af0d-c19460e39fe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7abeda1177474317a08d9ad4903bf0c3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:20.556025 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6dc3356c-0da8-41c6-af0d-c19460e39fe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7abeda1177474317a08d9ad4903bf0c3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:20.613153 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0a72d-44b8-43e9-98e2-8236d1c7939a admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:20.614548 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-2da0a72d-44b8-43e9-98e2-8236d1c7939a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:20.646666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6dc3356c-0da8-41c6-af0d-c19460e39fe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7abeda1177474317a08d9ad4903bf0c3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:20.647337 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6dc3356c-0da8-41c6-af0d-c19460e39fe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7abeda1177474317a08d9ad4903bf0c3'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:20.688560 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1411/2805] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:21:20 2022] DELETE /identity/v3/users/7abeda1177474317a08d9ad4903bf0c3 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:20.692868 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0a72d-44b8-43e9-98e2-8236d1c7939a admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:20.709046 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:20.755664 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba272fb-157c-45c5-8848-249cc4024d76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:20.756456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba272fb-157c-45c5-8848-249cc4024d76 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:20.761614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba272fb-157c-45c5-8848-249cc4024d76 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:20.762995 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba272fb-157c-45c5-8848-249cc4024d76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:20.767756 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-0ba272fb-157c-45c5-8848-249cc4024d76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:20.854762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba272fb-157c-45c5-8848-249cc4024d76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:20.856455 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba272fb-157c-45c5-8848-249cc4024d76 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:20.869093 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1395/2806] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:21:21.335858 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3356c-0da8-41c6-af0d-c19460e39fe9 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:21.337418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6dc3356c-0da8-41c6-af0d-c19460e39fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:21.362834 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3356c-0da8-41c6-af0d-c19460e39fe9 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:21.374641 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44de0ee-b493-43ee-9111-e02efb3dc499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:21.375496 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44de0ee-b493-43ee-9111-e02efb3dc499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:21.376199 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44de0ee-b493-43ee-9111-e02efb3dc499 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:21.376732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44de0ee-b493-43ee-9111-e02efb3dc499 admin admin] PATH_INFO: `/v3/projects/ac3716f045d64c9c9543e36c8f81bed3` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:21.379297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44de0ee-b493-43ee-9111-e02efb3dc499 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac3716f045d64c9c9543e36c8f81bed3)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:21.383474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44de0ee-b493-43ee-9111-e02efb3dc499 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:21.550179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-d44de0ee-b493-43ee-9111-e02efb3dc499 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac3716f045d64c9c9543e36c8f81bed3', 'request_id': 'req-d44de0ee-b493-43ee-9111-e02efb3dc499'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:21.560075 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-d44de0ee-b493-43ee-9111-e02efb3dc499 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac3716f045d64c9c9543e36c8f81bed3', 'request_id': 'req-d44de0ee-b493-43ee-9111-e02efb3dc499'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:21.574729 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-d44de0ee-b493-43ee-9111-e02efb3dc499 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac3716f045d64c9c9543e36c8f81bed3', 'request_id': 'req-d44de0ee-b493-43ee-9111-e02efb3dc499'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:21.684581 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-d44de0ee-b493-43ee-9111-e02efb3dc499 admin admin] The token cache is being invalidate because project ac3716f045d64c9c9543e36c8f81bed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:21:21.684869 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-d44de0ee-b493-43ee-9111-e02efb3dc499 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:21.685701 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-d44de0ee-b493-43ee-9111-e02efb3dc499 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:21.687530 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1412/2807] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:21:21 2022] DELETE /identity/v3/projects/ac3716f045d64c9c9543e36c8f81bed3 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:21.694104 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f96c81-e76f-4a6f-95e9-6c28a85a8663 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:21.694595 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f96c81-e76f-4a6f-95e9-6c28a85a8663 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:21.694943 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f96c81-e76f-4a6f-95e9-6c28a85a8663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:21.749954 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-c6f96c81-e76f-4a6f-95e9-6c28a85a8663 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:21.751292 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-c6f96c81-e76f-4a6f-95e9-6c28a85a8663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:21.849917 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1396/2808] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:21.859248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dae25f-6c69-48a4-bf35-6f1dcb4cb78e None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:21.859569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dae25f-6c69-48a4-bf35-6f1dcb4cb78e None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:21.859794 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dae25f-6c69-48a4-bf35-6f1dcb4cb78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:21.870185 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:21.886282 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-00dae25f-6c69-48a4-bf35-6f1dcb4cb78e None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:21.889460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-00dae25f-6c69-48a4-bf35-6f1dcb4cb78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:21.898736 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1413/2809] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:21.903012 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f96c81-e76f-4a6f-95e9-6c28a85a8663 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:21.923443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f96c81-e76f-4a6f-95e9-6c28a85a8663 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:21.933270 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89fadb-8a77-4ecc-a262-94f09f6f99a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:21.934105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89fadb-8a77-4ecc-a262-94f09f6f99a2 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:21.934600 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89fadb-8a77-4ecc-a262-94f09f6f99a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:21.935024 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89fadb-8a77-4ecc-a262-94f09f6f99a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:21.936009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89fadb-8a77-4ecc-a262-94f09f6f99a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:21:21.936664 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89fadb-8a77-4ecc-a262-94f09f6f99a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:21.938489 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89fadb-8a77-4ecc-a262-94f09f6f99a2 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:21.952449 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1397/2810] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:21:21.957380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dae25f-6c69-48a4-bf35-6f1dcb4cb78e None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:21.966845 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dae25f-6c69-48a4-bf35-6f1dcb4cb78e None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:21.971893 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872ac1d-fc98-4a6d-84d6-9e4ac24856dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:21.972703 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872ac1d-fc98-4a6d-84d6-9e4ac24856dc admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:21.977911 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872ac1d-fc98-4a6d-84d6-9e4ac24856dc admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:21.978189 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872ac1d-fc98-4a6d-84d6-9e4ac24856dc admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:21.978869 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872ac1d-fc98-4a6d-84d6-9e4ac24856dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:21.980794 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872ac1d-fc98-4a6d-84d6-9e4ac24856dc admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:22.087039 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1414/2811] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:21:21 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:21:22.091266 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89fadb-8a77-4ecc-a262-94f09f6f99a2 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:22.103786 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89fadb-8a77-4ecc-a262-94f09f6f99a2 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:22.221977 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=100 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:22.230662 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9694fe8-3f61-405a-8ab8-1a71a901aa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:22.231292 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9694fe8-3f61-405a-8ab8-1a71a901aa4e admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:22.231648 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9694fe8-3f61-405a-8ab8-1a71a901aa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:22.231925 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9694fe8-3f61-405a-8ab8-1a71a901aa4e admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:22.232602 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9694fe8-3f61-405a-8ab8-1a71a901aa4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:22.234676 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9694fe8-3f61-405a-8ab8-1a71a901aa4e admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:22.336248 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1398/2812] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:21:22 2022] POST /identity/v3/users => generated 287 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:21:22.340071 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872ac1d-fc98-4a6d-84d6-9e4ac24856dc admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:22.354409 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872ac1d-fc98-4a6d-84d6-9e4ac24856dc admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:22.363566 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d18d9-701c-49f0-a4e7-7ccb84c8a98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:22.363566 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d18d9-701c-49f0-a4e7-7ccb84c8a98f admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:22.363566 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d18d9-701c-49f0-a4e7-7ccb84c8a98f admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:22.363566 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d18d9-701c-49f0-a4e7-7ccb84c8a98f admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:22.364311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800d18d9-701c-49f0-a4e7-7ccb84c8a98f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:22.365912 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800d18d9-701c-49f0-a4e7-7ccb84c8a98f admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:22.379116 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1415/2813] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:22.383379 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9694fe8-3f61-405a-8ab8-1a71a901aa4e admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:22.398718 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9694fe8-3f61-405a-8ab8-1a71a901aa4e admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:22.410440 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffac98-2d54-49bf-820d-3b3aff164c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:22.411296 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffac98-2d54-49bf-820d-3b3aff164c2d admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:22.411994 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffac98-2d54-49bf-820d-3b3aff164c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:22.412464 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffac98-2d54-49bf-820d-3b3aff164c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:22.413432 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffac98-2d54-49bf-820d-3b3aff164c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:22.415288 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffac98-2d54-49bf-820d-3b3aff164c2d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:22.429390 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1399/2814] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:22.433977 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d18d9-701c-49f0-a4e7-7ccb84c8a98f admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:22.446650 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d18d9-701c-49f0-a4e7-7ccb84c8a98f admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:22.455168 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ee338-d0ce-4224-9d1d-05d6fa087127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:22.456126 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ee338-d0ce-4224-9d1d-05d6fa087127 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:22.457313 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ee338-d0ce-4224-9d1d-05d6fa087127 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:22.457591 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ee338-d0ce-4224-9d1d-05d6fa087127 admin admin] PATH_INFO: `/v3/projects/1446092faabf4ba580f7e4c64cf87ae8/users/9766b197c0704661bc273e5f1f1e401b/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:22.476058 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ee338-d0ce-4224-9d1d-05d6fa087127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1446092faabf4ba580f7e4c64cf87ae8, user_id=9766b197c0704661bc273e5f1f1e401b, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:22.479026 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ee338-d0ce-4224-9d1d-05d6fa087127 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:22.556688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1416/2815] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:21:22 2022] PUT /identity/v3/projects/1446092faabf4ba580f7e4c64cf87ae8/users/9766b197c0704661bc273e5f1f1e401b/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:22.569677 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56999b-bfd6-4dad-ac31-c6568ba738f1 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:22.570030 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56999b-bfd6-4dad-ac31-c6568ba738f1 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:22.570155 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56999b-bfd6-4dad-ac31-c6568ba738f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:22.618942 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-3f56999b-bfd6-4dad-ac31-c6568ba738f1 None None] MFA Rules not processed for user `9766b197c0704661bc273e5f1f1e401b`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:22.620256 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3f56999b-bfd6-4dad-ac31-c6568ba738f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:22.714010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1400/2816] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:22.727166 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ee338-d0ce-4224-9d1d-05d6fa087127 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:22.729103 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-2e6ee338-d0ce-4224-9d1d-05d6fa087127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:22.820813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ee338-d0ce-4224-9d1d-05d6fa087127 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:22.824539 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880bd40e-fb62-4921-a516-3512f93b1a0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:22.825144 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bd40e-fb62-4921-a516-3512f93b1a0a service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:22.825543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bd40e-fb62-4921-a516-3512f93b1a0a service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:22.825813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bd40e-fb62-4921-a516-3512f93b1a0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:22.835777 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880bd40e-fb62-4921-a516-3512f93b1a0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:22.837366 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880bd40e-fb62-4921-a516-3512f93b1a0a service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:22.849676 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1417/2817] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:21:22.872806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56999b-bfd6-4dad-ac31-c6568ba738f1 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:22.949969 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56999b-bfd6-4dad-ac31-c6568ba738f1 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:22.956478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3905503-e9ac-4fc0-a198-3de9be44427e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:22.957652 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3905503-e9ac-4fc0-a198-3de9be44427e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:22.958002 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3905503-e9ac-4fc0-a198-3de9be44427e admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:22.958236 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3905503-e9ac-4fc0-a198-3de9be44427e admin admin] PATH_INFO: `/v3/users/9766b197c0704661bc273e5f1f1e401b` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:22.959542 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3905503-e9ac-4fc0-a198-3de9be44427e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9766b197c0704661bc273e5f1f1e401b)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:22.964406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3905503-e9ac-4fc0-a198-3de9be44427e admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:23.219916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-a3905503-e9ac-4fc0-a198-3de9be44427e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9766b197c0704661bc273e5f1f1e401b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:23.236185 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-a3905503-e9ac-4fc0-a198-3de9be44427e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9766b197c0704661bc273e5f1f1e401b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:23.237100 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-a3905503-e9ac-4fc0-a198-3de9be44427e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9766b197c0704661bc273e5f1f1e401b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:23.252936 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:23.263546 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-a3905503-e9ac-4fc0-a198-3de9be44427e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9766b197c0704661bc273e5f1f1e401b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:23.264380 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-a3905503-e9ac-4fc0-a198-3de9be44427e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9766b197c0704661bc273e5f1f1e401b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:23.379373 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1401/2818] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:21:22 2022] DELETE /identity/v3/users/9766b197c0704661bc273e5f1f1e401b => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:23.381254 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f2e61-0f72-4ab8-91b1-b2e3b84a519c None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:23.382049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f2e61-0f72-4ab8-91b1-b2e3b84a519c None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:23.382470 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f2e61-0f72-4ab8-91b1-b2e3b84a519c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:23.419601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-816f2e61-0f72-4ab8-91b1-b2e3b84a519c None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:23.422315 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-816f2e61-0f72-4ab8-91b1-b2e3b84a519c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:23.529526 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1418/2819] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:23.530584 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3905503-e9ac-4fc0-a198-3de9be44427e admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:23.532220 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a3905503-e9ac-4fc0-a198-3de9be44427e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:23.627186 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3905503-e9ac-4fc0-a198-3de9be44427e admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:23.632050 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9bc96-b00e-4a16-9feb-a34c3e945fd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:23.632996 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9bc96-b00e-4a16-9feb-a34c3e945fd4 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:23.633512 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9bc96-b00e-4a16-9feb-a34c3e945fd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:23.633876 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9bc96-b00e-4a16-9feb-a34c3e945fd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:23.646406 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9bc96-b00e-4a16-9feb-a34c3e945fd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:23.648586 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9bc96-b00e-4a16-9feb-a34c3e945fd4 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:23.662715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1402/2820] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:21:24.432234 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f2e61-0f72-4ab8-91b1-b2e3b84a519c None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:24.434886 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-816f2e61-0f72-4ab8-91b1-b2e3b84a519c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:24.464536 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f2e61-0f72-4ab8-91b1-b2e3b84a519c None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:24.486452 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e4245-63f0-481e-ba8d-05ed114df020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:24.487377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e4245-63f0-481e-ba8d-05ed114df020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:24.487650 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e4245-63f0-481e-ba8d-05ed114df020 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:24.487956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e4245-63f0-481e-ba8d-05ed114df020 admin admin] PATH_INFO: `/v3/projects/1446092faabf4ba580f7e4c64cf87ae8` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:24.489651 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e4245-63f0-481e-ba8d-05ed114df020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1446092faabf4ba580f7e4c64cf87ae8)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:24.491654 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e4245-63f0-481e-ba8d-05ed114df020 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:24.590059 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:24.599894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-bd6e4245-63f0-481e-ba8d-05ed114df020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1446092faabf4ba580f7e4c64cf87ae8', 'request_id': 'req-bd6e4245-63f0-481e-ba8d-05ed114df020'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:24.607004 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-bd6e4245-63f0-481e-ba8d-05ed114df020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1446092faabf4ba580f7e4c64cf87ae8', 'request_id': 'req-bd6e4245-63f0-481e-ba8d-05ed114df020'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:24.612928 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-bd6e4245-63f0-481e-ba8d-05ed114df020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1446092faabf4ba580f7e4c64cf87ae8', 'request_id': 'req-bd6e4245-63f0-481e-ba8d-05ed114df020'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:24.696036 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-bd6e4245-63f0-481e-ba8d-05ed114df020 admin admin] The token cache is being invalidate because project 1446092faabf4ba580f7e4c64cf87ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:21:24.696538 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-bd6e4245-63f0-481e-ba8d-05ed114df020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:24.697354 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-bd6e4245-63f0-481e-ba8d-05ed114df020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:24.700206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1419/2821] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:21:24 2022] DELETE /identity/v3/projects/1446092faabf4ba580f7e4c64cf87ae8 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:26.217444 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9bc96-b00e-4a16-9feb-a34c3e945fd4 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:26.218961 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6eb9bc96-b00e-4a16-9feb-a34c3e945fd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:26.329791 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9bc96-b00e-4a16-9feb-a34c3e945fd4 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:26.400918 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a854f6-23af-46ce-ab09-57905c4ca146 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:26.401635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a854f6-23af-46ce-ab09-57905c4ca146 service placement] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:26.401923 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a854f6-23af-46ce-ab09-57905c4ca146 service placement] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:26.402190 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a854f6-23af-46ce-ab09-57905c4ca146 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:26.403922 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-42a854f6-23af-46ce-ab09-57905c4ca146 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:26.419481 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a854f6-23af-46ce-ab09-57905c4ca146 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:26.422908 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a854f6-23af-46ce-ab09-57905c4ca146 service placement] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:26.432320 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1403/2822] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:21:26.446126 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e4245-63f0-481e-ba8d-05ed114df020 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:26.447761 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-bd6e4245-63f0-481e-ba8d-05ed114df020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:26.505259 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabeee5-b56f-4882-9be7-10b9e5fdeb64 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:26.505541 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabeee5-b56f-4882-9be7-10b9e5fdeb64 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:26.505808 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabeee5-b56f-4882-9be7-10b9e5fdeb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:26.537722 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-7dabeee5-b56f-4882-9be7-10b9e5fdeb64 None None] MFA Rules not processed for user `d3acbdab00c64d84871545062a5b5311`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:26.538934 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-7dabeee5-b56f-4882-9be7-10b9e5fdeb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:26.544927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:26.559863 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e4245-63f0-481e-ba8d-05ed114df020 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:26.565705 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062dff7-3643-4b9d-ac91-8d68c424ce06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:26.567969 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062dff7-3643-4b9d-ac91-8d68c424ce06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:26.568281 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062dff7-3643-4b9d-ac91-8d68c424ce06 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:26.568562 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062dff7-3643-4b9d-ac91-8d68c424ce06 admin admin] PATH_INFO: `/v3/users/9c2fdfdb62b54f05b7d81f14ec197cca` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:26.569733 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062dff7-3643-4b9d-ac91-8d68c424ce06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c2fdfdb62b54f05b7d81f14ec197cca)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:26.571775 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062dff7-3643-4b9d-ac91-8d68c424ce06 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:26.608854 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1404/2823] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:21:26.613835 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabeee5-b56f-4882-9be7-10b9e5fdeb64 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:26.615372 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-7dabeee5-b56f-4882-9be7-10b9e5fdeb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:26.628525 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabeee5-b56f-4882-9be7-10b9e5fdeb64 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:26.634556 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a2b5e-e74a-47ec-86f7-5e00caf44e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:26.635180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a2b5e-e74a-47ec-86f7-5e00caf44e20 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:26.635458 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a2b5e-e74a-47ec-86f7-5e00caf44e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:26.635724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a2b5e-e74a-47ec-86f7-5e00caf44e20 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:26.636391 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a2b5e-e74a-47ec-86f7-5e00caf44e20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:26.638293 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a2b5e-e74a-47ec-86f7-5e00caf44e20 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:26.768927 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1405/2824] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:21:26 2022] POST /identity/v3/projects => generated 371 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:21:26.772816 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a2b5e-e74a-47ec-86f7-5e00caf44e20 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:26.803265 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b062dff7-3643-4b9d-ac91-8d68c424ce06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c2fdfdb62b54f05b7d81f14ec197cca'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:26.803943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b062dff7-3643-4b9d-ac91-8d68c424ce06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c2fdfdb62b54f05b7d81f14ec197cca'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:26.813376 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b062dff7-3643-4b9d-ac91-8d68c424ce06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c2fdfdb62b54f05b7d81f14ec197cca'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:26.839656 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a2b5e-e74a-47ec-86f7-5e00caf44e20 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:26.844712 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760c1ed-fed2-4b4b-bde6-99bb3e2ce491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:26.845949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760c1ed-fed2-4b4b-bde6-99bb3e2ce491 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:26.845949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760c1ed-fed2-4b4b-bde6-99bb3e2ce491 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:26.845949 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760c1ed-fed2-4b4b-bde6-99bb3e2ce491 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:26.846574 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760c1ed-fed2-4b4b-bde6-99bb3e2ce491 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:26.848611 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760c1ed-fed2-4b4b-bde6-99bb3e2ce491 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:26.895725 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b062dff7-3643-4b9d-ac91-8d68c424ce06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c2fdfdb62b54f05b7d81f14ec197cca'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:26.896374 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-b062dff7-3643-4b9d-ac91-8d68c424ce06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c2fdfdb62b54f05b7d81f14ec197cca'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:26.927007 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1420/2825] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:21:26 2022] DELETE /identity/v3/users/9c2fdfdb62b54f05b7d81f14ec197cca => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:26.931665 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1406/2826] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:21:26 2022] POST /identity/v3/users => generated 292 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:21:26.935188 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062dff7-3643-4b9d-ac91-8d68c424ce06 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:26.939404 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-b062dff7-3643-4b9d-ac91-8d68c424ce06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:26.999324 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:27.019489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062dff7-3643-4b9d-ac91-8d68c424ce06 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:27.025034 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33dfd01-3e6b-4620-9699-904b595a6003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:27.025650 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dfd01-3e6b-4620-9699-904b595a6003 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:27.025926 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dfd01-3e6b-4620-9699-904b595a6003 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:27.026197 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dfd01-3e6b-4620-9699-904b595a6003 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:27.026837 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33dfd01-3e6b-4620-9699-904b595a6003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:27.028461 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33dfd01-3e6b-4620-9699-904b595a6003 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:27.041109 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1421/2827] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:27.045140 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760c1ed-fed2-4b4b-bde6-99bb3e2ce491 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:27.059284 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760c1ed-fed2-4b4b-bde6-99bb3e2ce491 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:27.064654 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1367b-c83d-4ff4-87c8-f8532de17546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:27.065456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1367b-c83d-4ff4-87c8-f8532de17546 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:27.065839 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1367b-c83d-4ff4-87c8-f8532de17546 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:27.066225 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1367b-c83d-4ff4-87c8-f8532de17546 admin admin] PATH_INFO: `/v3/projects/4af31fd1c14e4db18aee028aebe3edd1/users/1d0c4f3cfb594a13821d1082b07cbbc4/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:27.088685 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1367b-c83d-4ff4-87c8-f8532de17546 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4af31fd1c14e4db18aee028aebe3edd1, user_id=1d0c4f3cfb594a13821d1082b07cbbc4, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:27.093023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1367b-c83d-4ff4-87c8-f8532de17546 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:27.172740 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1407/2828] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:21:27 2022] PUT /identity/v3/projects/4af31fd1c14e4db18aee028aebe3edd1/users/1d0c4f3cfb594a13821d1082b07cbbc4/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:21:27.181623 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33dfd01-3e6b-4620-9699-904b595a6003 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:27.193197 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33dfd01-3e6b-4620-9699-904b595a6003 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:27.197570 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1367b-c83d-4ff4-87c8-f8532de17546 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:27.201634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-1cb1367b-c83d-4ff4-87c8-f8532de17546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:27.274064 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:27.276766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59faec9e-68e2-4dd7-989c-f52248a3da4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:27.277385 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59faec9e-68e2-4dd7-989c-f52248a3da4d admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:27.295768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59faec9e-68e2-4dd7-989c-f52248a3da4d admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:27.295768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59faec9e-68e2-4dd7-989c-f52248a3da4d admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:27.295768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59faec9e-68e2-4dd7-989c-f52248a3da4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:27.295768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59faec9e-68e2-4dd7-989c-f52248a3da4d admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:27.296325 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1367b-c83d-4ff4-87c8-f8532de17546 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:27.296399 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1422/2829] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:27.296484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79963bb-d950-41ad-8141-225dc5bb5992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:27.296484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79963bb-d950-41ad-8141-225dc5bb5992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:27.296484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79963bb-d950-41ad-8141-225dc5bb5992 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:27.296484 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79963bb-d950-41ad-8141-225dc5bb5992 admin admin] PATH_INFO: `/v3/projects/a826c462a3fa429eb866d354afcbe950` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:27.296723 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59faec9e-68e2-4dd7-989c-f52248a3da4d admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:27.297643 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79963bb-d950-41ad-8141-225dc5bb5992 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a826c462a3fa429eb866d354afcbe950)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:27.299499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79963bb-d950-41ad-8141-225dc5bb5992 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:27.307943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59faec9e-68e2-4dd7-989c-f52248a3da4d admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:27.314424 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111ae70-b8ef-4665-af4e-52b2b4b3e9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:27.315206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111ae70-b8ef-4665-af4e-52b2b4b3e9bf admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:27.315553 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111ae70-b8ef-4665-af4e-52b2b4b3e9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:27.315950 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111ae70-b8ef-4665-af4e-52b2b4b3e9bf admin admin] PATH_INFO: `/v3/projects/4af31fd1c14e4db18aee028aebe3edd1/users/1d0c4f3cfb594a13821d1082b07cbbc4/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:27.318902 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111ae70-b8ef-4665-af4e-52b2b4b3e9bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4af31fd1c14e4db18aee028aebe3edd1, user_id=1d0c4f3cfb594a13821d1082b07cbbc4, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:27.322015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111ae70-b8ef-4665-af4e-52b2b4b3e9bf admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:27.334085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1423/2830] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:21:27 2022] PUT /identity/v3/projects/4af31fd1c14e4db18aee028aebe3edd1/users/1d0c4f3cfb594a13821d1082b07cbbc4/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:27.412774 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e79963bb-d950-41ad-8141-225dc5bb5992 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a826c462a3fa429eb866d354afcbe950', 'request_id': 'req-e79963bb-d950-41ad-8141-225dc5bb5992'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:27.475689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e79963bb-d950-41ad-8141-225dc5bb5992 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a826c462a3fa429eb866d354afcbe950', 'request_id': 'req-e79963bb-d950-41ad-8141-225dc5bb5992'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:27.482303 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e79963bb-d950-41ad-8141-225dc5bb5992 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a826c462a3fa429eb866d354afcbe950', 'request_id': 'req-e79963bb-d950-41ad-8141-225dc5bb5992'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:27.499496 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e79963bb-d950-41ad-8141-225dc5bb5992 admin admin] The token cache is being invalidate because project a826c462a3fa429eb866d354afcbe950 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:21:27.499781 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e79963bb-d950-41ad-8141-225dc5bb5992 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:27.501346 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-e79963bb-d950-41ad-8141-225dc5bb5992 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:27.509547 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1408/2831] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:21:27 2022] DELETE /identity/v3/projects/a826c462a3fa429eb866d354afcbe950 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:21:27.512671 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473ffc7-b2da-4410-a0fa-721acaab150f None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:27.513121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473ffc7-b2da-4410-a0fa-721acaab150f None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:27.513195 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473ffc7-b2da-4410-a0fa-721acaab150f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:27.538133 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-8473ffc7-b2da-4410-a0fa-721acaab150f None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:27.539355 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8473ffc7-b2da-4410-a0fa-721acaab150f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:27.602083 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1424/2832] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:27.609810 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff1313-16c2-4ff1-b2c7-5925f46ca55a None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:27.610248 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff1313-16c2-4ff1-b2c7-5925f46ca55a None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:27.610447 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff1313-16c2-4ff1-b2c7-5925f46ca55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:27.631814 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-7dff1313-16c2-4ff1-b2c7-5925f46ca55a None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:27.633208 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-7dff1313-16c2-4ff1-b2c7-5925f46ca55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:27.644714 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1409/2833] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:27.647253 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473ffc7-b2da-4410-a0fa-721acaab150f None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:27.668880 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473ffc7-b2da-4410-a0fa-721acaab150f None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:27.674234 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5b7e5-8662-482d-ae05-98623dc37237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:27.675045 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5b7e5-8662-482d-ae05-98623dc37237 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:27.675326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5b7e5-8662-482d-ae05-98623dc37237 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:27.675686 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5b7e5-8662-482d-ae05-98623dc37237 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:27.676475 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d5b7e5-8662-482d-ae05-98623dc37237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:21:27.676785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d5b7e5-8662-482d-ae05-98623dc37237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:27.678430 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d5b7e5-8662-482d-ae05-98623dc37237 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:27.690555 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1425/2834] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:21:27.695626 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff1313-16c2-4ff1-b2c7-5925f46ca55a None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:27.704914 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff1313-16c2-4ff1-b2c7-5925f46ca55a None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:27.710275 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be776b2d-2ffa-4bcb-a23f-32359eaea46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:27.710881 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be776b2d-2ffa-4bcb-a23f-32359eaea46d admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:27.711160 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be776b2d-2ffa-4bcb-a23f-32359eaea46d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:27.711430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be776b2d-2ffa-4bcb-a23f-32359eaea46d admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:27.712063 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be776b2d-2ffa-4bcb-a23f-32359eaea46d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:27.713991 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be776b2d-2ffa-4bcb-a23f-32359eaea46d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:27.856330 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1410/2835] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:21:27 2022] POST /identity/v3/projects => generated 367 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:21:27.857574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5b7e5-8662-482d-ae05-98623dc37237 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:27.869421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5b7e5-8662-482d-ae05-98623dc37237 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:27.921548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:27.932095 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c16fbd-235b-487f-8720-f72408e9fd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:27.932768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c16fbd-235b-487f-8720-f72408e9fd96 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:27.933046 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c16fbd-235b-487f-8720-f72408e9fd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:27.933317 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c16fbd-235b-487f-8720-f72408e9fd96 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:27.933951 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c16fbd-235b-487f-8720-f72408e9fd96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:27.935899 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c16fbd-235b-487f-8720-f72408e9fd96 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:28.045094 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1426/2836] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:21:27 2022] POST /identity/v3/users => generated 284 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:21:28.049255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be776b2d-2ffa-4bcb-a23f-32359eaea46d admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:28.060681 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be776b2d-2ffa-4bcb-a23f-32359eaea46d admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:28.065306 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0788b-c318-4092-93b1-fd30d36d6fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:28.073028 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0788b-c318-4092-93b1-fd30d36d6fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:28.073362 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0788b-c318-4092-93b1-fd30d36d6fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:28.073639 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0788b-c318-4092-93b1-fd30d36d6fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:28.074324 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0788b-c318-4092-93b1-fd30d36d6fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:28.079756 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0788b-c318-4092-93b1-fd30d36d6fd8 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:28.096793 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1411/2837] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:21:28.101187 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c16fbd-235b-487f-8720-f72408e9fd96 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:28.111766 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c16fbd-235b-487f-8720-f72408e9fd96 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:28.117560 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac7223-9a46-4230-bfe4-50ae18d31d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:28.119932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac7223-9a46-4230-bfe4-50ae18d31d5e admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:28.119932 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac7223-9a46-4230-bfe4-50ae18d31d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:28.126681 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac7223-9a46-4230-bfe4-50ae18d31d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:28.126681 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac7223-9a46-4230-bfe4-50ae18d31d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:28.126681 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac7223-9a46-4230-bfe4-50ae18d31d5e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:28.137299 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1427/2838] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:21:28.140911 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0788b-c318-4092-93b1-fd30d36d6fd8 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:28.154236 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0788b-c318-4092-93b1-fd30d36d6fd8 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:28.160020 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fae85-1d34-4559-af92-5d6c3d350e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:28.161050 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fae85-1d34-4559-af92-5d6c3d350e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:28.161402 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fae85-1d34-4559-af92-5d6c3d350e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:28.161751 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fae85-1d34-4559-af92-5d6c3d350e7e admin admin] PATH_INFO: `/v3/projects/0ad08dbe0e564c5a9167c76896c648bb/users/b7d3c65f794d4f04bc64905950cdf4b1/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:28.180543 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fae85-1d34-4559-af92-5d6c3d350e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad08dbe0e564c5a9167c76896c648bb, user_id=b7d3c65f794d4f04bc64905950cdf4b1, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:28.183456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fae85-1d34-4559-af92-5d6c3d350e7e admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:28.249919 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1412/2839] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:21:28 2022] PUT /identity/v3/projects/0ad08dbe0e564c5a9167c76896c648bb/users/b7d3c65f794d4f04bc64905950cdf4b1/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:28.251952 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59f226-f264-436a-8cde-a6df76785934 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:28.252328 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59f226-f264-436a-8cde-a6df76785934 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:28.252661 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59f226-f264-436a-8cde-a6df76785934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:28.281874 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-cb59f226-f264-436a-8cde-a6df76785934 None None] MFA Rules not processed for user `b7d3c65f794d4f04bc64905950cdf4b1`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:28.283098 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-cb59f226-f264-436a-8cde-a6df76785934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:28.356956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1428/2840] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:28.368819 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fae85-1d34-4559-af92-5d6c3d350e7e admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:28.372624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-e47fae85-1d34-4559-af92-5d6c3d350e7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:28.480261 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fae85-1d34-4559-af92-5d6c3d350e7e admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:28.484052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93591f3e-2f0b-40ad-8a4a-f5b542a960bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:28.484706 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93591f3e-2f0b-40ad-8a4a-f5b542a960bb service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:28.485006 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93591f3e-2f0b-40ad-8a4a-f5b542a960bb service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:28.485283 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93591f3e-2f0b-40ad-8a4a-f5b542a960bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:28.495255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93591f3e-2f0b-40ad-8a4a-f5b542a960bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:28.496766 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93591f3e-2f0b-40ad-8a4a-f5b542a960bb service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:28.507569 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1413/2841] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:21:33.383479 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7067057-057f-4221-95ce-a26ba2f396a8 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:33.392909 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7067057-057f-4221-95ce-a26ba2f396a8 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:33.392909 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7067057-057f-4221-95ce-a26ba2f396a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:33.413069 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-c7067057-057f-4221-95ce-a26ba2f396a8 None None] MFA Rules not processed for user `1d0c4f3cfb594a13821d1082b07cbbc4`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:33.414304 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-c7067057-057f-4221-95ce-a26ba2f396a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:33.519214 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1429/2842] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:33.535271 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93591f3e-2f0b-40ad-8a4a-f5b542a960bb service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:33.538023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-93591f3e-2f0b-40ad-8a4a-f5b542a960bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:33.562053 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:33.617832 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93591f3e-2f0b-40ad-8a4a-f5b542a960bb service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:33.636296 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594bfd5-5dde-4544-849f-bc4646d3e116 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:33.636296 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594bfd5-5dde-4544-849f-bc4646d3e116 service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:33.636296 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594bfd5-5dde-4544-849f-bc4646d3e116 service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:33.636296 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594bfd5-5dde-4544-849f-bc4646d3e116 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:33.638307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-1594bfd5-5dde-4544-849f-bc4646d3e116 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:33.686430 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594bfd5-5dde-4544-849f-bc4646d3e116 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:33.688048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594bfd5-5dde-4544-849f-bc4646d3e116 service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:33.698099 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1414/2843] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:21:36.561943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da67ba6-5c59-4af8-95e2-9935c8a6d67f None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:36.562455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da67ba6-5c59-4af8-95e2-9935c8a6d67f None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:36.562543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da67ba6-5c59-4af8-95e2-9935c8a6d67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:36.581947 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-4da67ba6-5c59-4af8-95e2-9935c8a6d67f None None] MFA Rules not processed for user `d3acbdab00c64d84871545062a5b5311`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:36.583149 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-4da67ba6-5c59-4af8-95e2-9935c8a6d67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:36.591734 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1430/2844] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:39.362646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594bfd5-5dde-4544-849f-bc4646d3e116 service nova] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:39.425141 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594bfd5-5dde-4544-849f-bc4646d3e116 service nova] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:39.429647 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d9853-6acc-4bf9-ac95-a0d2fb4a90bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:39.430423 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d9853-6acc-4bf9-ac95-a0d2fb4a90bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:39.430699 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d9853-6acc-4bf9-ac95-a0d2fb4a90bb admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:39.430966 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d9853-6acc-4bf9-ac95-a0d2fb4a90bb admin admin] PATH_INFO: `/v3/users/b7d3c65f794d4f04bc64905950cdf4b1` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:39.432077 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d9853-6acc-4bf9-ac95-a0d2fb4a90bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7d3c65f794d4f04bc64905950cdf4b1)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:39.433999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d9853-6acc-4bf9-ac95-a0d2fb4a90bb admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:39.621506 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-251d9853-6acc-4bf9-ac95-a0d2fb4a90bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7d3c65f794d4f04bc64905950cdf4b1'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:39.628774 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-251d9853-6acc-4bf9-ac95-a0d2fb4a90bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7d3c65f794d4f04bc64905950cdf4b1'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:39.629388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-251d9853-6acc-4bf9-ac95-a0d2fb4a90bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7d3c65f794d4f04bc64905950cdf4b1'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:39.631327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:39.642499 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-251d9853-6acc-4bf9-ac95-a0d2fb4a90bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7d3c65f794d4f04bc64905950cdf4b1'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:39.643219 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-251d9853-6acc-4bf9-ac95-a0d2fb4a90bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7d3c65f794d4f04bc64905950cdf4b1'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:39.709555 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1415/2845] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:21:39 2022] DELETE /identity/v3/users/b7d3c65f794d4f04bc64905950cdf4b1 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:39.714726 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f1a4b-1528-4524-89e7-ce033ee973cc None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:39.716346 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f1a4b-1528-4524-89e7-ce033ee973cc None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:39.716346 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f1a4b-1528-4524-89e7-ce033ee973cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:39.734123 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-100f1a4b-1528-4524-89e7-ce033ee973cc None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:39.735344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-100f1a4b-1528-4524-89e7-ce033ee973cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:39.771319 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:39.797586 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1431/2846] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:39.807411 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d9853-6acc-4bf9-ac95-a0d2fb4a90bb admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:39.815087 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-251d9853-6acc-4bf9-ac95-a0d2fb4a90bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:39.873782 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d9853-6acc-4bf9-ac95-a0d2fb4a90bb admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:39.877655 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b924075-0780-440c-b3cd-dc5beb2d101b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:39.878260 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b924075-0780-440c-b3cd-dc5beb2d101b service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:39.878539 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b924075-0780-440c-b3cd-dc5beb2d101b service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:39.878810 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b924075-0780-440c-b3cd-dc5beb2d101b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:39.888537 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b924075-0780-440c-b3cd-dc5beb2d101b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:39.889985 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b924075-0780-440c-b3cd-dc5beb2d101b service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:39.900016 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1416/2847] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:21:40.332445 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f1a4b-1528-4524-89e7-ce033ee973cc None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:40.334014 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-100f1a4b-1528-4524-89e7-ce033ee973cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:40.347300 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f1a4b-1528-4524-89e7-ce033ee973cc None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:40.352971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb6d2a-dba7-4e35-97c8-6b2ffeb13d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:40.353612 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb6d2a-dba7-4e35-97c8-6b2ffeb13d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:40.353887 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb6d2a-dba7-4e35-97c8-6b2ffeb13d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:40.354157 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb6d2a-dba7-4e35-97c8-6b2ffeb13d4e admin admin] PATH_INFO: `/v3/projects/0ad08dbe0e564c5a9167c76896c648bb` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:40.355299 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb6d2a-dba7-4e35-97c8-6b2ffeb13d4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ad08dbe0e564c5a9167c76896c648bb)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:40.357398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb6d2a-dba7-4e35-97c8-6b2ffeb13d4e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:40.464360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-19cb6d2a-dba7-4e35-97c8-6b2ffeb13d4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ad08dbe0e564c5a9167c76896c648bb', 'request_id': 'req-19cb6d2a-dba7-4e35-97c8-6b2ffeb13d4e'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:40.470541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-19cb6d2a-dba7-4e35-97c8-6b2ffeb13d4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ad08dbe0e564c5a9167c76896c648bb', 'request_id': 'req-19cb6d2a-dba7-4e35-97c8-6b2ffeb13d4e'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:40.476294 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-19cb6d2a-dba7-4e35-97c8-6b2ffeb13d4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ad08dbe0e564c5a9167c76896c648bb', 'request_id': 'req-19cb6d2a-dba7-4e35-97c8-6b2ffeb13d4e'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:40.546580 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-19cb6d2a-dba7-4e35-97c8-6b2ffeb13d4e admin admin] The token cache is being invalidate because project 0ad08dbe0e564c5a9167c76896c648bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:21:40.546874 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-19cb6d2a-dba7-4e35-97c8-6b2ffeb13d4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:40.548097 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-19cb6d2a-dba7-4e35-97c8-6b2ffeb13d4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:40.549662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1432/2848] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:21:40 2022] DELETE /identity/v3/projects/0ad08dbe0e564c5a9167c76896c648bb => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:40.554719 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7409a80-4bbf-490e-a1ea-74f303ffc791 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:40.555327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7409a80-4bbf-490e-a1ea-74f303ffc791 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:40.555327 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7409a80-4bbf-490e-a1ea-74f303ffc791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:40.576277 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-f7409a80-4bbf-490e-a1ea-74f303ffc791 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:40.577646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f7409a80-4bbf-490e-a1ea-74f303ffc791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:40.651166 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1417/2849] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:40.658840 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b7a1d-df58-4f85-aad2-7685ce736982 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:40.659122 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b7a1d-df58-4f85-aad2-7685ce736982 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:40.659674 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b7a1d-df58-4f85-aad2-7685ce736982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:40.679515 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-ae0b7a1d-df58-4f85-aad2-7685ce736982 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:40.680821 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-ae0b7a1d-df58-4f85-aad2-7685ce736982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:40.688786 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1433/2850] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:40.693002 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7409a80-4bbf-490e-a1ea-74f303ffc791 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:40.703724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7409a80-4bbf-490e-a1ea-74f303ffc791 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:40.708392 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225b809-3cff-497c-8fe3-41de5821dd09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:40.708997 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225b809-3cff-497c-8fe3-41de5821dd09 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:40.709264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225b809-3cff-497c-8fe3-41de5821dd09 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:40.709667 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225b809-3cff-497c-8fe3-41de5821dd09 admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:40.710308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225b809-3cff-497c-8fe3-41de5821dd09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:21:40.710604 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225b809-3cff-497c-8fe3-41de5821dd09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:40.712952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225b809-3cff-497c-8fe3-41de5821dd09 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:40.724544 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1418/2851] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:21:40.729326 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b7a1d-df58-4f85-aad2-7685ce736982 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:40.738624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b7a1d-df58-4f85-aad2-7685ce736982 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:40.744222 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f590b-526a-482e-895e-ea3943e42ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:40.744975 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f590b-526a-482e-895e-ea3943e42ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:40.745474 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f590b-526a-482e-895e-ea3943e42ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:40.745835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f590b-526a-482e-895e-ea3943e42ae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:40.746542 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f590b-526a-482e-895e-ea3943e42ae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:40.748595 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f590b-526a-482e-895e-ea3943e42ae4 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:40.843471 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1434/2852] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:21:40 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:21:40.847528 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225b809-3cff-497c-8fe3-41de5821dd09 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:40.858818 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225b809-3cff-497c-8fe3-41de5821dd09 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:40.916568 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f2a58-f56b-4cf8-9a3e-c4e82b9aa9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:40.917177 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f2a58-f56b-4cf8-9a3e-c4e82b9aa9a7 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:40.917457 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f2a58-f56b-4cf8-9a3e-c4e82b9aa9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:40.917729 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f2a58-f56b-4cf8-9a3e-c4e82b9aa9a7 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:40.918363 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f2a58-f56b-4cf8-9a3e-c4e82b9aa9a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:40.920307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f2a58-f56b-4cf8-9a3e-c4e82b9aa9a7 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:40.932229 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:41.058200 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1419/2853] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:21:40 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:21:41.062158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f590b-526a-482e-895e-ea3943e42ae4 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:41.073131 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f590b-526a-482e-895e-ea3943e42ae4 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:41.077995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47efceca-5993-412b-a3a0-beb2bd16a3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:41.078393 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efceca-5993-412b-a3a0-beb2bd16a3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:41.078664 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efceca-5993-412b-a3a0-beb2bd16a3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:41.078929 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efceca-5993-412b-a3a0-beb2bd16a3f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:41.079574 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47efceca-5993-412b-a3a0-beb2bd16a3f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:41.081145 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47efceca-5993-412b-a3a0-beb2bd16a3f3 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:41.090822 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1435/2854] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:41.094729 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f2a58-f56b-4cf8-9a3e-c4e82b9aa9a7 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:41.105240 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f2a58-f56b-4cf8-9a3e-c4e82b9aa9a7 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:41.110076 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d08ab10-2f6e-409e-aff6-c30377fb22d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:41.110718 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08ab10-2f6e-409e-aff6-c30377fb22d6 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:41.111035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08ab10-2f6e-409e-aff6-c30377fb22d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:41.111302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08ab10-2f6e-409e-aff6-c30377fb22d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:41.111931 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d08ab10-2f6e-409e-aff6-c30377fb22d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:41.113572 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d08ab10-2f6e-409e-aff6-c30377fb22d6 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:41.125363 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1420/2855] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:41.129532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47efceca-5993-412b-a3a0-beb2bd16a3f3 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:41.139777 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47efceca-5993-412b-a3a0-beb2bd16a3f3 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:41.144540 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cc6a8-d0f5-4aa1-8129-9c5061df1cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:41.145208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cc6a8-d0f5-4aa1-8129-9c5061df1cc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:41.145483 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cc6a8-d0f5-4aa1-8129-9c5061df1cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:41.145751 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cc6a8-d0f5-4aa1-8129-9c5061df1cc5 admin admin] PATH_INFO: `/v3/projects/1649b11f458141f8b47bbf42eb19e876/users/86d4d3fb610842ad9ac1a12e5248d7c1/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:41.163513 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887cc6a8-d0f5-4aa1-8129-9c5061df1cc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1649b11f458141f8b47bbf42eb19e876, user_id=86d4d3fb610842ad9ac1a12e5248d7c1, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:41.166699 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887cc6a8-d0f5-4aa1-8129-9c5061df1cc5 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:41.233876 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1436/2856] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:21:41 2022] PUT /identity/v3/projects/1649b11f458141f8b47bbf42eb19e876/users/86d4d3fb610842ad9ac1a12e5248d7c1/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:41.245658 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f3792-89c1-4a41-87c2-5c3e881624b7 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:41.245938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f3792-89c1-4a41-87c2-5c3e881624b7 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:41.246207 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f3792-89c1-4a41-87c2-5c3e881624b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:41.274126 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-878f3792-89c1-4a41-87c2-5c3e881624b7 None None] MFA Rules not processed for user `86d4d3fb610842ad9ac1a12e5248d7c1`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:41.275339 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-878f3792-89c1-4a41-87c2-5c3e881624b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:41.341930 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1421/2857] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:21:41.353460 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cc6a8-d0f5-4aa1-8129-9c5061df1cc5 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:41.355145 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-887cc6a8-d0f5-4aa1-8129-9c5061df1cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:41.417278 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:41.433893 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cc6a8-d0f5-4aa1-8129-9c5061df1cc5 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:41.439452 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66297662-111a-4e46-a7b9-795b83a51955 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:41.443950 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66297662-111a-4e46-a7b9-795b83a51955 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:41.443950 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66297662-111a-4e46-a7b9-795b83a51955 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:41.443950 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66297662-111a-4e46-a7b9-795b83a51955 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:41.448558 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66297662-111a-4e46-a7b9-795b83a51955 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:41.450121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66297662-111a-4e46-a7b9-795b83a51955 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:41.458890 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1437/2858] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:21:41.671823 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f3792-89c1-4a41-87c2-5c3e881624b7 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:41.739384 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f3792-89c1-4a41-87c2-5c3e881624b7 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:41.744229 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3fe03-017b-4e75-b121-e7f281bf9371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:41.744916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3fe03-017b-4e75-b121-e7f281bf9371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:41.745189 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3fe03-017b-4e75-b121-e7f281bf9371 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:41.745452 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3fe03-017b-4e75-b121-e7f281bf9371 admin admin] PATH_INFO: `/v3/users/86d4d3fb610842ad9ac1a12e5248d7c1` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:41.746554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3fe03-017b-4e75-b121-e7f281bf9371 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86d4d3fb610842ad9ac1a12e5248d7c1)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:41.748512 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3fe03-017b-4e75-b121-e7f281bf9371 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:41.936756 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-22b3fe03-017b-4e75-b121-e7f281bf9371 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86d4d3fb610842ad9ac1a12e5248d7c1'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:41.943673 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-22b3fe03-017b-4e75-b121-e7f281bf9371 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86d4d3fb610842ad9ac1a12e5248d7c1'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:41.944420 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-22b3fe03-017b-4e75-b121-e7f281bf9371 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86d4d3fb610842ad9ac1a12e5248d7c1'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:41.957321 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-22b3fe03-017b-4e75-b121-e7f281bf9371 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86d4d3fb610842ad9ac1a12e5248d7c1'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:41.958010 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-22b3fe03-017b-4e75-b121-e7f281bf9371 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86d4d3fb610842ad9ac1a12e5248d7c1'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:41.960620 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:42.076549 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1422/2859] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:21:41 2022] DELETE /identity/v3/users/86d4d3fb610842ad9ac1a12e5248d7c1 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:42.080913 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70fbfd-13d5-4680-a96c-23bccab9e063 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:42.081402 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70fbfd-13d5-4680-a96c-23bccab9e063 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:42.081492 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70fbfd-13d5-4680-a96c-23bccab9e063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:42.100894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-1b70fbfd-13d5-4680-a96c-23bccab9e063 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:42.102170 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-1b70fbfd-13d5-4680-a96c-23bccab9e063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:42.178318 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1438/2860] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:42.188477 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3fe03-017b-4e75-b121-e7f281bf9371 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:42.189871 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-22b3fe03-017b-4e75-b121-e7f281bf9371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:42.248596 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3fe03-017b-4e75-b121-e7f281bf9371 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:42.252375 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63195a-652c-4f0a-9cf7-1083cd6e76c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:42.252971 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63195a-652c-4f0a-9cf7-1083cd6e76c7 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:42.253245 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63195a-652c-4f0a-9cf7-1083cd6e76c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:42.253513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63195a-652c-4f0a-9cf7-1083cd6e76c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:42.263505 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63195a-652c-4f0a-9cf7-1083cd6e76c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:42.264970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63195a-652c-4f0a-9cf7-1083cd6e76c7 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:42.274420 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1423/2861] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:21:42.717464 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70fbfd-13d5-4680-a96c-23bccab9e063 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:42.722183 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-1b70fbfd-13d5-4680-a96c-23bccab9e063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:42.732747 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70fbfd-13d5-4680-a96c-23bccab9e063 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:42.738041 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf0fbc-41cc-4a34-9f61-5c553a525639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:42.738660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf0fbc-41cc-4a34-9f61-5c553a525639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:42.738933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf0fbc-41cc-4a34-9f61-5c553a525639 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:42.739286 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf0fbc-41cc-4a34-9f61-5c553a525639 admin admin] PATH_INFO: `/v3/projects/1649b11f458141f8b47bbf42eb19e876` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:42.741048 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf0fbc-41cc-4a34-9f61-5c553a525639 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1649b11f458141f8b47bbf42eb19e876)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:42.744353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf0fbc-41cc-4a34-9f61-5c553a525639 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:42.865375 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6cdf0fbc-41cc-4a34-9f61-5c553a525639 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1649b11f458141f8b47bbf42eb19e876', 'request_id': 'req-6cdf0fbc-41cc-4a34-9f61-5c553a525639'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:42.872206 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6cdf0fbc-41cc-4a34-9f61-5c553a525639 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1649b11f458141f8b47bbf42eb19e876', 'request_id': 'req-6cdf0fbc-41cc-4a34-9f61-5c553a525639'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:42.877671 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6cdf0fbc-41cc-4a34-9f61-5c553a525639 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1649b11f458141f8b47bbf42eb19e876', 'request_id': 'req-6cdf0fbc-41cc-4a34-9f61-5c553a525639'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:42.949823 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6cdf0fbc-41cc-4a34-9f61-5c553a525639 admin admin] The token cache is being invalidate because project 1649b11f458141f8b47bbf42eb19e876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:21:42.950049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6cdf0fbc-41cc-4a34-9f61-5c553a525639 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:42.950674 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-6cdf0fbc-41cc-4a34-9f61-5c553a525639 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:42.952176 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1439/2862] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:21:42 2022] DELETE /identity/v3/projects/1649b11f458141f8b47bbf42eb19e876 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:42.957153 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35db354-6aa8-4b6d-bced-f78d2a335605 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:42.957601 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35db354-6aa8-4b6d-bced-f78d2a335605 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:42.957704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35db354-6aa8-4b6d-bced-f78d2a335605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:42.978363 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-f35db354-6aa8-4b6d-bced-f78d2a335605 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:42.979592 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f35db354-6aa8-4b6d-bced-f78d2a335605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:43.037232 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1424/2863] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:43.045939 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f4f4f-1b29-46f3-9614-eb66060ea46b None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:43.046242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f4f4f-1b29-46f3-9614-eb66060ea46b None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:43.046534 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f4f4f-1b29-46f3-9614-eb66060ea46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:43.075503 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-138f4f4f-1b29-46f3-9614-eb66060ea46b None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:43.076748 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-138f4f4f-1b29-46f3-9614-eb66060ea46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:43.087086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1440/2864] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:43.091572 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35db354-6aa8-4b6d-bced-f78d2a335605 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:43.102279 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35db354-6aa8-4b6d-bced-f78d2a335605 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:43.106598 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e01fc5-8e0a-4a05-9816-75799b6f9301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:43.107198 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e01fc5-8e0a-4a05-9816-75799b6f9301 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:43.107474 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e01fc5-8e0a-4a05-9816-75799b6f9301 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:43.107741 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e01fc5-8e0a-4a05-9816-75799b6f9301 admin admin] PATH_INFO: `/v3/domains` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:43.108414 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e01fc5-8e0a-4a05-9816-75799b6f9301 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:21:43.108711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e01fc5-8e0a-4a05-9816-75799b6f9301 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:43.110281 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e01fc5-8e0a-4a05-9816-75799b6f9301 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:43.122496 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1425/2865] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:21:43.126689 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f4f4f-1b29-46f3-9614-eb66060ea46b None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:43.136639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f4f4f-1b29-46f3-9614-eb66060ea46b None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:43.141481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab423ddf-5ec5-4113-b018-0268171c50f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:43.142085 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab423ddf-5ec5-4113-b018-0268171c50f4 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:43.142363 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab423ddf-5ec5-4113-b018-0268171c50f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:43.142634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab423ddf-5ec5-4113-b018-0268171c50f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:43.143315 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab423ddf-5ec5-4113-b018-0268171c50f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:43.145267 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab423ddf-5ec5-4113-b018-0268171c50f4 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:43.242827 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1441/2866] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:21:43 2022] POST /identity/v3/projects => generated 405 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:21:43.247271 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e01fc5-8e0a-4a05-9816-75799b6f9301 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:43.261684 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e01fc5-8e0a-4a05-9816-75799b6f9301 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:43.321672 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febeeb02-4ac5-45fd-a281-c77fad43ff36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:43.322281 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febeeb02-4ac5-45fd-a281-c77fad43ff36 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:43.322559 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febeeb02-4ac5-45fd-a281-c77fad43ff36 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:43.322828 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febeeb02-4ac5-45fd-a281-c77fad43ff36 admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:43.323491 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febeeb02-4ac5-45fd-a281-c77fad43ff36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:43.325482 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febeeb02-4ac5-45fd-a281-c77fad43ff36 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:43.433663 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1426/2867] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:21:43 2022] POST /identity/v3/users => generated 303 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:21:43.437776 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab423ddf-5ec5-4113-b018-0268171c50f4 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:43.448829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab423ddf-5ec5-4113-b018-0268171c50f4 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:43.453541 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cb3f7-2238-480b-b633-64eea320fa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:43.454234 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cb3f7-2238-480b-b633-64eea320fa43 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:43.454439 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cb3f7-2238-480b-b633-64eea320fa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:43.454712 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cb3f7-2238-480b-b633-64eea320fa43 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:43.455351 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8cb3f7-2238-480b-b633-64eea320fa43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:43.457016 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8cb3f7-2238-480b-b633-64eea320fa43 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:43.467971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1442/2868] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:43.472019 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febeeb02-4ac5-45fd-a281-c77fad43ff36 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:43.482206 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febeeb02-4ac5-45fd-a281-c77fad43ff36 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:43.487312 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4512eb5-6831-4c6d-b95c-52d3c142a124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:43.487968 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4512eb5-6831-4c6d-b95c-52d3c142a124 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:43.488361 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4512eb5-6831-4c6d-b95c-52d3c142a124 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:43.488628 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4512eb5-6831-4c6d-b95c-52d3c142a124 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:43.489283 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4512eb5-6831-4c6d-b95c-52d3c142a124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:43.491023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4512eb5-6831-4c6d-b95c-52d3c142a124 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:43.503662 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1427/2869] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:43.507774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cb3f7-2238-480b-b633-64eea320fa43 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:43.518750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cb3f7-2238-480b-b633-64eea320fa43 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:43.523344 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f8f1ae-7a0f-496b-966c-174b5bf0acc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:43.524570 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8f1ae-7a0f-496b-966c-174b5bf0acc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:43.524966 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8f1ae-7a0f-496b-966c-174b5bf0acc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:43.525241 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8f1ae-7a0f-496b-966c-174b5bf0acc8 admin admin] PATH_INFO: `/v3/projects/a14d14c6ffe541919c98b50739751b47/users/dd66ae92fa8f47e98d78920f5e9f723b/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:43.539983 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f8f1ae-7a0f-496b-966c-174b5bf0acc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a14d14c6ffe541919c98b50739751b47, user_id=dd66ae92fa8f47e98d78920f5e9f723b, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:43.542657 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f8f1ae-7a0f-496b-966c-174b5bf0acc8 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:43.633977 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1443/2870] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:21:43 2022] PUT /identity/v3/projects/a14d14c6ffe541919c98b50739751b47/users/dd66ae92fa8f47e98d78920f5e9f723b/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:43.644254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced11d5f-25c2-41ff-92c0-284330890da6 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:43.644532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced11d5f-25c2-41ff-92c0-284330890da6 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:43.644806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced11d5f-25c2-41ff-92c0-284330890da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:43.652456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:43.672784 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-ced11d5f-25c2-41ff-92c0-284330890da6 None None] MFA Rules not processed for user `dd66ae92fa8f47e98d78920f5e9f723b`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:43.673976 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ced11d5f-25c2-41ff-92c0-284330890da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:43.743330 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1428/2871] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:43 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:43.747951 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f8f1ae-7a0f-496b-966c-174b5bf0acc8 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:43.753674 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:43.811237 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f8f1ae-7a0f-496b-966c-174b5bf0acc8 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:43.816502 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafbb1f-b0a8-482a-8520-b9b07bf2d02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:43.817100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafbb1f-b0a8-482a-8520-b9b07bf2d02b admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:43.817377 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafbb1f-b0a8-482a-8520-b9b07bf2d02b admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:43.817644 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafbb1f-b0a8-482a-8520-b9b07bf2d02b admin admin] PATH_INFO: `/v3/projects` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:43.818288 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dafbb1f-b0a8-482a-8520-b9b07bf2d02b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:43.820172 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dafbb1f-b0a8-482a-8520-b9b07bf2d02b admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:43.920594 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1444/2872] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:21:43 2022] POST /identity/v3/projects => generated 405 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:21:43.924739 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced11d5f-25c2-41ff-92c0-284330890da6 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:43.936756 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced11d5f-25c2-41ff-92c0-284330890da6 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:43.999127 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58072659-68c8-48a6-9c3c-d8a4a30fc3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:43.999747 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58072659-68c8-48a6-9c3c-d8a4a30fc3fb admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:43.999916 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58072659-68c8-48a6-9c3c-d8a4a30fc3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:44.000217 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58072659-68c8-48a6-9c3c-d8a4a30fc3fb admin admin] PATH_INFO: `/v3/users` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:44.000847 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58072659-68c8-48a6-9c3c-d8a4a30fc3fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:44.002743 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58072659-68c8-48a6-9c3c-d8a4a30fc3fb admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:44.098854 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1429/2873] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:21:43 2022] POST /identity/v3/users => generated 309 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:21:44.103383 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafbb1f-b0a8-482a-8520-b9b07bf2d02b admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:44.113566 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafbb1f-b0a8-482a-8520-b9b07bf2d02b admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:44.119109 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52564c3b-3ba9-40ee-ad2a-e050da71ca9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:44.119700 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52564c3b-3ba9-40ee-ad2a-e050da71ca9f admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:44.119984 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52564c3b-3ba9-40ee-ad2a-e050da71ca9f admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:44.120290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52564c3b-3ba9-40ee-ad2a-e050da71ca9f admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:44.120923 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52564c3b-3ba9-40ee-ad2a-e050da71ca9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:44.122514 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52564c3b-3ba9-40ee-ad2a-e050da71ca9f admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:44.134125 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1445/2874] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:21:44.138331 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58072659-68c8-48a6-9c3c-d8a4a30fc3fb admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:44.148232 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58072659-68c8-48a6-9c3c-d8a4a30fc3fb admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:44.153997 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1e816-039a-4e2b-858e-d1faef376187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:44.154785 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1e816-039a-4e2b-858e-d1faef376187 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:44.155121 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1e816-039a-4e2b-858e-d1faef376187 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:44.155401 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1e816-039a-4e2b-858e-d1faef376187 admin admin] PATH_INFO: `/v3/projects/56281e846db04e799aeef8791ceb9bd6/users/43317c4d812d4f309630d582c67ac2ac/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:44.173170 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1e816-039a-4e2b-858e-d1faef376187 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56281e846db04e799aeef8791ceb9bd6, user_id=43317c4d812d4f309630d582c67ac2ac, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:44.175929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1e816-039a-4e2b-858e-d1faef376187 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:44.235668 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1430/2875] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:21:44 2022] PUT /identity/v3/projects/56281e846db04e799aeef8791ceb9bd6/users/43317c4d812d4f309630d582c67ac2ac/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:44.239814 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52564c3b-3ba9-40ee-ad2a-e050da71ca9f admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:44.250612 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52564c3b-3ba9-40ee-ad2a-e050da71ca9f admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:44.308713 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117dbed-32ec-4752-b8c0-459f3b568c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:44.309322 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117dbed-32ec-4752-b8c0-459f3b568c32 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:44.309596 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117dbed-32ec-4752-b8c0-459f3b568c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:44.309868 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117dbed-32ec-4752-b8c0-459f3b568c32 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:44.310504 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117dbed-32ec-4752-b8c0-459f3b568c32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:44.312080 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117dbed-32ec-4752-b8c0-459f3b568c32 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:44.323088 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1446/2876] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:44.327058 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1e816-039a-4e2b-858e-d1faef376187 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:44.337674 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1e816-039a-4e2b-858e-d1faef376187 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:44.343394 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe948cea-939a-4cd3-9638-cc5050ebfd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:44.344285 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe948cea-939a-4cd3-9638-cc5050ebfd3a admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:44.344656 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe948cea-939a-4cd3-9638-cc5050ebfd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:44.344933 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe948cea-939a-4cd3-9638-cc5050ebfd3a admin admin] PATH_INFO: `/v3/projects/56281e846db04e799aeef8791ceb9bd6/users/43317c4d812d4f309630d582c67ac2ac/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:44.347568 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe948cea-939a-4cd3-9638-cc5050ebfd3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=56281e846db04e799aeef8791ceb9bd6, user_id=43317c4d812d4f309630d582c67ac2ac, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:44.350812 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe948cea-939a-4cd3-9638-cc5050ebfd3a admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:44.368041 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1431/2877] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:21:44 2022] PUT /identity/v3/projects/56281e846db04e799aeef8791ceb9bd6/users/43317c4d812d4f309630d582c67ac2ac/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:44.379123 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f00a0-7473-4199-82a1-aee66e2a7b30 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:44.379408 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f00a0-7473-4199-82a1-aee66e2a7b30 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:44.379697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f00a0-7473-4199-82a1-aee66e2a7b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:44.407619 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-841f00a0-7473-4199-82a1-aee66e2a7b30 None None] MFA Rules not processed for user `43317c4d812d4f309630d582c67ac2ac`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:44.408963 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-841f00a0-7473-4199-82a1-aee66e2a7b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:44.481515 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:44.485630 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1447/2878] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:44 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:44.496500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe948cea-939a-4cd3-9638-cc5050ebfd3a admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:44.498167 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-fe948cea-939a-4cd3-9638-cc5050ebfd3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:44.547435 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:44.558130 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe948cea-939a-4cd3-9638-cc5050ebfd3a admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:44.561824 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7768e8e-ada9-45c3-baf5-5347b5f08628 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:44.562415 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7768e8e-ada9-45c3-baf5-5347b5f08628 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:44.562689 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7768e8e-ada9-45c3-baf5-5347b5f08628 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:44.562952 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7768e8e-ada9-45c3-baf5-5347b5f08628 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:44.572371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7768e8e-ada9-45c3-baf5-5347b5f08628 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:44.573779 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7768e8e-ada9-45c3-baf5-5347b5f08628 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:44.583452 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1432/2879] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:44 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:21:44.725779 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f00a0-7473-4199-82a1-aee66e2a7b30 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:44.787666 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f00a0-7473-4199-82a1-aee66e2a7b30 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:44.793315 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad729761-cad0-49a1-bc41-bfe207702f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:44.793952 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad729761-cad0-49a1-bc41-bfe207702f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:44.794223 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad729761-cad0-49a1-bc41-bfe207702f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:44.794489 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad729761-cad0-49a1-bc41-bfe207702f54 admin admin] PATH_INFO: `/v3/users/dd66ae92fa8f47e98d78920f5e9f723b` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:44.795604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad729761-cad0-49a1-bc41-bfe207702f54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd66ae92fa8f47e98d78920f5e9f723b)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:44.797611 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad729761-cad0-49a1-bc41-bfe207702f54 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:44.956536 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ad729761-cad0-49a1-bc41-bfe207702f54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd66ae92fa8f47e98d78920f5e9f723b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:44.957190 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ad729761-cad0-49a1-bc41-bfe207702f54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd66ae92fa8f47e98d78920f5e9f723b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:44.965029 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ad729761-cad0-49a1-bc41-bfe207702f54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd66ae92fa8f47e98d78920f5e9f723b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:45.041553 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ad729761-cad0-49a1-bc41-bfe207702f54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd66ae92fa8f47e98d78920f5e9f723b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:45.042178 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ad729761-cad0-49a1-bc41-bfe207702f54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd66ae92fa8f47e98d78920f5e9f723b'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:45.062370 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1448/2880] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:21:44 2022] DELETE /identity/v3/users/dd66ae92fa8f47e98d78920f5e9f723b => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:45.066935 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fce2f-2a9f-4e9d-ab44-a3fed0b25d1c None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:45.067302 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fce2f-2a9f-4e9d-ab44-a3fed0b25d1c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:45.067487 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fce2f-2a9f-4e9d-ab44-a3fed0b25d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:45.088063 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-154fce2f-2a9f-4e9d-ab44-a3fed0b25d1c None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:45.089441 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-154fce2f-2a9f-4e9d-ab44-a3fed0b25d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:45.149482 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1433/2881] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:21:45.159219 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad729761-cad0-49a1-bc41-bfe207702f54 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:45.160995 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-ad729761-cad0-49a1-bc41-bfe207702f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:45.221802 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad729761-cad0-49a1-bc41-bfe207702f54 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:45.225501 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b6c77-e39d-42d7-a2eb-599d37c5c2b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:45.226106 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b6c77-e39d-42d7-a2eb-599d37c5c2b3 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:45.226420 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b6c77-e39d-42d7-a2eb-599d37c5c2b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:45.226688 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b6c77-e39d-42d7-a2eb-599d37c5c2b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:45.235576 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b6c77-e39d-42d7-a2eb-599d37c5c2b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:45.237017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b6c77-e39d-42d7-a2eb-599d37c5c2b3 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:45.246928 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1449/2882] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:21:45.721098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154fce2f-2a9f-4e9d-ab44-a3fed0b25d1c None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:45.722502 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-154fce2f-2a9f-4e9d-ab44-a3fed0b25d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:45.737191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154fce2f-2a9f-4e9d-ab44-a3fed0b25d1c None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:45.741986 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2764c-edb8-43e4-8cda-ff37066737f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:45.743337 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2764c-edb8-43e4-8cda-ff37066737f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:45.743684 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2764c-edb8-43e4-8cda-ff37066737f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:45.744021 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2764c-edb8-43e4-8cda-ff37066737f6 admin admin] PATH_INFO: `/v3/projects/a14d14c6ffe541919c98b50739751b47` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:45.745404 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2764c-edb8-43e4-8cda-ff37066737f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a14d14c6ffe541919c98b50739751b47)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:45.747553 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2764c-edb8-43e4-8cda-ff37066737f6 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:45.891403 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-59c2764c-edb8-43e4-8cda-ff37066737f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a14d14c6ffe541919c98b50739751b47', 'request_id': 'req-59c2764c-edb8-43e4-8cda-ff37066737f6'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:45.933164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-59c2764c-edb8-43e4-8cda-ff37066737f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a14d14c6ffe541919c98b50739751b47', 'request_id': 'req-59c2764c-edb8-43e4-8cda-ff37066737f6'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:45.939243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-59c2764c-edb8-43e4-8cda-ff37066737f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a14d14c6ffe541919c98b50739751b47', 'request_id': 'req-59c2764c-edb8-43e4-8cda-ff37066737f6'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:45.950229 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-59c2764c-edb8-43e4-8cda-ff37066737f6 admin admin] The token cache is being invalidate because project a14d14c6ffe541919c98b50739751b47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:21:45.950517 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-59c2764c-edb8-43e4-8cda-ff37066737f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:45.951153 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-59c2764c-edb8-43e4-8cda-ff37066737f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:45.952666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1434/2883] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:21:45 2022] DELETE /identity/v3/projects/a14d14c6ffe541919c98b50739751b47 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:45.956785 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b6c77-e39d-42d7-a2eb-599d37c5c2b3 service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:45.966979 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b6c77-e39d-42d7-a2eb-599d37c5c2b3 service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:45.985974 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:46.023395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23557f-1387-4dda-a804-de031c05f7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:46.024575 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23557f-1387-4dda-a804-de031c05f7ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:46.024852 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23557f-1387-4dda-a804-de031c05f7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:46.025122 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23557f-1387-4dda-a804-de031c05f7ff admin admin] PATH_INFO: `/v3/users/43317c4d812d4f309630d582c67ac2ac` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:46.026432 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23557f-1387-4dda-a804-de031c05f7ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=43317c4d812d4f309630d582c67ac2ac)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:46.029016 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23557f-1387-4dda-a804-de031c05f7ff admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:46.192527 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ad23557f-1387-4dda-a804-de031c05f7ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43317c4d812d4f309630d582c67ac2ac'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:46.193200 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ad23557f-1387-4dda-a804-de031c05f7ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43317c4d812d4f309630d582c67ac2ac'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:46.200905 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ad23557f-1387-4dda-a804-de031c05f7ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43317c4d812d4f309630d582c67ac2ac'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:46.274191 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ad23557f-1387-4dda-a804-de031c05f7ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43317c4d812d4f309630d582c67ac2ac'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:46.275051 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-ad23557f-1387-4dda-a804-de031c05f7ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43317c4d812d4f309630d582c67ac2ac'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:46.294283 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1450/2884] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:21:45 2022] DELETE /identity/v3/users/43317c4d812d4f309630d582c67ac2ac => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:46.740993 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2764c-edb8-43e4-8cda-ff37066737f6 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:46.744098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-59c2764c-edb8-43e4-8cda-ff37066737f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:46.765124 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2764c-edb8-43e4-8cda-ff37066737f6 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:46.829566 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ec0b8-fe49-44b1-be54-2ae40eccae5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:46.830199 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ec0b8-fe49-44b1-be54-2ae40eccae5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:46.830470 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ec0b8-fe49-44b1-be54-2ae40eccae5b admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:46.830733 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ec0b8-fe49-44b1-be54-2ae40eccae5b admin admin] PATH_INFO: `/v3/projects/56281e846db04e799aeef8791ceb9bd6` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:46.831950 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ec0b8-fe49-44b1-be54-2ae40eccae5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=56281e846db04e799aeef8791ceb9bd6)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:46.833890 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ec0b8-fe49-44b1-be54-2ae40eccae5b admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:46.846035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:46.961675 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1e2ec0b8-fe49-44b1-be54-2ae40eccae5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56281e846db04e799aeef8791ceb9bd6', 'request_id': 'req-1e2ec0b8-fe49-44b1-be54-2ae40eccae5b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:47.036038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1e2ec0b8-fe49-44b1-be54-2ae40eccae5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56281e846db04e799aeef8791ceb9bd6', 'request_id': 'req-1e2ec0b8-fe49-44b1-be54-2ae40eccae5b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:47.042014 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1e2ec0b8-fe49-44b1-be54-2ae40eccae5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56281e846db04e799aeef8791ceb9bd6', 'request_id': 'req-1e2ec0b8-fe49-44b1-be54-2ae40eccae5b'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:47.053708 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1e2ec0b8-fe49-44b1-be54-2ae40eccae5b admin admin] The token cache is being invalidate because project 56281e846db04e799aeef8791ceb9bd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:21:47.053999 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1e2ec0b8-fe49-44b1-be54-2ae40eccae5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:47.054660 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-1e2ec0b8-fe49-44b1-be54-2ae40eccae5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:21:47.056693 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1435/2885] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:21:46 2022] DELETE /identity/v3/projects/56281e846db04e799aeef8791ceb9bd6 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:47.061613 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec109bf5-65fa-4e78-b243-d03adf4527e5 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:47.061916 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec109bf5-65fa-4e78-b243-d03adf4527e5 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:47.062182 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec109bf5-65fa-4e78-b243-d03adf4527e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:47.082113 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-ec109bf5-65fa-4e78-b243-d03adf4527e5 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:47.083289 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-ec109bf5-65fa-4e78-b243-d03adf4527e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:47.142614 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1451/2886] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:47.152118 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9201d7a-f979-4325-bf26-c66eff73814f None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:47.152523 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9201d7a-f979-4325-bf26-c66eff73814f None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:47.152682 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9201d7a-f979-4325-bf26-c66eff73814f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:47.175897 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-b9201d7a-f979-4325-bf26-c66eff73814f None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:47.177167 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-b9201d7a-f979-4325-bf26-c66eff73814f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:47.185931 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1436/2887] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:47.190170 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec109bf5-65fa-4e78-b243-d03adf4527e5 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:47.200031 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec109bf5-65fa-4e78-b243-d03adf4527e5 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:47.204660 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f911cc-9d03-4dec-9bf2-260381c500bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:47.205276 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f911cc-9d03-4dec-9bf2-260381c500bc admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:47.205554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f911cc-9d03-4dec-9bf2-260381c500bc admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:47.205823 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f911cc-9d03-4dec-9bf2-260381c500bc admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:47.206461 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f911cc-9d03-4dec-9bf2-260381c500bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:21:47.206782 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f911cc-9d03-4dec-9bf2-260381c500bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:47.208537 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f911cc-9d03-4dec-9bf2-260381c500bc admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:47.220380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1452/2888] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:21:47.224428 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9201d7a-f979-4325-bf26-c66eff73814f None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:47.234059 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9201d7a-f979-4325-bf26-c66eff73814f None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:47.238646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ee750-438b-4005-884a-68fc911a5931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:47.239228 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ee750-438b-4005-884a-68fc911a5931 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:47.239497 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ee750-438b-4005-884a-68fc911a5931 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:47.239760 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ee750-438b-4005-884a-68fc911a5931 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:47.240420 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ee750-438b-4005-884a-68fc911a5931 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:47.242282 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ee750-438b-4005-884a-68fc911a5931 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:47.363960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1437/2889] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:21:47 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:21:47.368997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f911cc-9d03-4dec-9bf2-260381c500bc admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:47.379100 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f911cc-9d03-4dec-9bf2-260381c500bc admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:47.405597 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:47.438958 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70203cba-a638-448b-9c07-071e18e6b457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:47.439571 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70203cba-a638-448b-9c07-071e18e6b457 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:47.439852 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70203cba-a638-448b-9c07-071e18e6b457 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:47.440121 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70203cba-a638-448b-9c07-071e18e6b457 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:47.440772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70203cba-a638-448b-9c07-071e18e6b457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:47.442658 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70203cba-a638-448b-9c07-071e18e6b457 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:47.536805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1453/2890] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:21:47 2022] POST /identity/v3/users => generated 291 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 23:21:47.541139 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ee750-438b-4005-884a-68fc911a5931 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:47.551366 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ee750-438b-4005-884a-68fc911a5931 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:47.556572 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ef079-3a8b-41f3-a24a-7fc779ec6f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:47.558387 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ef079-3a8b-41f3-a24a-7fc779ec6f28 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:47.558387 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ef079-3a8b-41f3-a24a-7fc779ec6f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:47.558387 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ef079-3a8b-41f3-a24a-7fc779ec6f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:47.558387 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ef079-3a8b-41f3-a24a-7fc779ec6f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:47.559898 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ef079-3a8b-41f3-a24a-7fc779ec6f28 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:47.571164 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1438/2891] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:47.575610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70203cba-a638-448b-9c07-071e18e6b457 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:47.586317 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70203cba-a638-448b-9c07-071e18e6b457 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:47.591633 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0cc60-c54d-4a25-98ab-39d1535cebc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:47.592272 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0cc60-c54d-4a25-98ab-39d1535cebc6 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:47.592549 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0cc60-c54d-4a25-98ab-39d1535cebc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:47.592814 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0cc60-c54d-4a25-98ab-39d1535cebc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:47.593452 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0cc60-c54d-4a25-98ab-39d1535cebc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:47.595087 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0cc60-c54d-4a25-98ab-39d1535cebc6 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:47.605247 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1454/2892] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:47.609573 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ef079-3a8b-41f3-a24a-7fc779ec6f28 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:47.621581 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ef079-3a8b-41f3-a24a-7fc779ec6f28 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:47.624691 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f948b3f-6a03-409c-8288-a37f5fbfbb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:47.625326 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f948b3f-6a03-409c-8288-a37f5fbfbb39 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:47.625598 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f948b3f-6a03-409c-8288-a37f5fbfbb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:47.625860 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f948b3f-6a03-409c-8288-a37f5fbfbb39 admin admin] PATH_INFO: `/v3/projects/c7b8e0ea82a14e1a86c3bc9610edd1bd/users/13a21a526f31480f878a697f82c3c9de/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:47.641769 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f948b3f-6a03-409c-8288-a37f5fbfbb39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7b8e0ea82a14e1a86c3bc9610edd1bd, user_id=13a21a526f31480f878a697f82c3c9de, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:47.644433 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f948b3f-6a03-409c-8288-a37f5fbfbb39 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:47.725126 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1439/2893] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:21:47 2022] PUT /identity/v3/projects/c7b8e0ea82a14e1a86c3bc9610edd1bd/users/13a21a526f31480f878a697f82c3c9de/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:21:47.735624 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4cd78-76ac-4de3-9fde-64458528c188 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:47.735896 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4cd78-76ac-4de3-9fde-64458528c188 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:47.736189 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4cd78-76ac-4de3-9fde-64458528c188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:47.765805 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-02b4cd78-76ac-4de3-9fde-64458528c188 None None] MFA Rules not processed for user `13a21a526f31480f878a697f82c3c9de`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:47.766987 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-02b4cd78-76ac-4de3-9fde-64458528c188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:47.839998 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1455/2894] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:47.844486 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f948b3f-6a03-409c-8288-a37f5fbfbb39 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:47.905852 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f948b3f-6a03-409c-8288-a37f5fbfbb39 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:47.910761 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4822ed-97e8-419a-a415-ebd99fdc544d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:47.911359 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4822ed-97e8-419a-a415-ebd99fdc544d admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:47.911710 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4822ed-97e8-419a-a415-ebd99fdc544d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:47.911892 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4822ed-97e8-419a-a415-ebd99fdc544d admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:47.912619 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4822ed-97e8-419a-a415-ebd99fdc544d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:47.919191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4822ed-97e8-419a-a415-ebd99fdc544d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:47.996787 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1440/2895] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:21:47 2022] POST /identity/v3/projects => generated 381 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:21:48.000718 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4cd78-76ac-4de3-9fde-64458528c188 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:48.011904 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4cd78-76ac-4de3-9fde-64458528c188 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:48.070686 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d758f-9069-4bdd-8277-66f5bc49c92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:48.071298 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d758f-9069-4bdd-8277-66f5bc49c92a admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:48.071577 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d758f-9069-4bdd-8277-66f5bc49c92a admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:48.071850 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d758f-9069-4bdd-8277-66f5bc49c92a admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:48.072508 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d758f-9069-4bdd-8277-66f5bc49c92a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:48.074412 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d758f-9069-4bdd-8277-66f5bc49c92a admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:48.177086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1456/2896] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:21:47 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:21:48.181800 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4822ed-97e8-419a-a415-ebd99fdc544d admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:48.192271 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4822ed-97e8-419a-a415-ebd99fdc544d admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:48.197154 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb5d28-cd30-4b97-b3ca-8ae69331e7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:48.197748 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb5d28-cd30-4b97-b3ca-8ae69331e7eb admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:48.198068 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb5d28-cd30-4b97-b3ca-8ae69331e7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:48.203766 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb5d28-cd30-4b97-b3ca-8ae69331e7eb admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:48.203766 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb5d28-cd30-4b97-b3ca-8ae69331e7eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:48.203766 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb5d28-cd30-4b97-b3ca-8ae69331e7eb admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:48.211602 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1441/2897] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:21:48.215894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d758f-9069-4bdd-8277-66f5bc49c92a admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:48.226399 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d758f-9069-4bdd-8277-66f5bc49c92a admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:48.231732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a8b7e-b874-40e8-96b2-f7b414688c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:48.232410 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a8b7e-b874-40e8-96b2-f7b414688c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:48.232679 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a8b7e-b874-40e8-96b2-f7b414688c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:48.232943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a8b7e-b874-40e8-96b2-f7b414688c7c admin admin] PATH_INFO: `/v3/projects/0e5ac67c44da4bfd9c9c85f144e86b23/users/885c052c69ae48369fdae8647e8613b4/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:48.247515 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a8b7e-b874-40e8-96b2-f7b414688c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e5ac67c44da4bfd9c9c85f144e86b23, user_id=885c052c69ae48369fdae8647e8613b4, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:48.250145 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a8b7e-b874-40e8-96b2-f7b414688c7c admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:48.321697 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1457/2898] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:21:48 2022] PUT /identity/v3/projects/0e5ac67c44da4bfd9c9c85f144e86b23/users/885c052c69ae48369fdae8647e8613b4/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:48.325815 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb5d28-cd30-4b97-b3ca-8ae69331e7eb admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:48.335994 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb5d28-cd30-4b97-b3ca-8ae69331e7eb admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:48.344255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:48.393953 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa62a4-f35b-46f5-9ddd-07534ca39eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:48.394710 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa62a4-f35b-46f5-9ddd-07534ca39eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:48.395039 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa62a4-f35b-46f5-9ddd-07534ca39eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:48.395374 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa62a4-f35b-46f5-9ddd-07534ca39eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:48.396116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa62a4-f35b-46f5-9ddd-07534ca39eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:48.397826 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa62a4-f35b-46f5-9ddd-07534ca39eb9 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:48.411424 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1442/2899] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:21:48.414937 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a8b7e-b874-40e8-96b2-f7b414688c7c admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:48.420589 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:48.426050 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a8b7e-b874-40e8-96b2-f7b414688c7c admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:48.431135 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1a27c-10e9-4920-b10b-bf4e9d226789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:48.431724 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1a27c-10e9-4920-b10b-bf4e9d226789 admin admin] REQUEST_METHOD: `PUT` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:48.432008 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1a27c-10e9-4920-b10b-bf4e9d226789 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:48.432310 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1a27c-10e9-4920-b10b-bf4e9d226789 admin admin] PATH_INFO: `/v3/projects/0e5ac67c44da4bfd9c9c85f144e86b23/users/885c052c69ae48369fdae8647e8613b4/roles/a836108447dc439e994611bc74d53d9c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:48.434390 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1a27c-10e9-4920-b10b-bf4e9d226789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e5ac67c44da4bfd9c9c85f144e86b23, user_id=885c052c69ae48369fdae8647e8613b4, role_id=a836108447dc439e994611bc74d53d9c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:48.437053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1a27c-10e9-4920-b10b-bf4e9d226789 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:48.447486 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1458/2900] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:21:48 2022] PUT /identity/v3/projects/0e5ac67c44da4bfd9c9c85f144e86b23/users/885c052c69ae48369fdae8647e8613b4/roles/a836108447dc439e994611bc74d53d9c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:21:48.459093 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee854a-e4f6-458d-83db-2501b45ca27d None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:48.463112 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee854a-e4f6-458d-83db-2501b45ca27d None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:48.463112 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee854a-e4f6-458d-83db-2501b45ca27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:48.488513 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-29ee854a-e4f6-458d-83db-2501b45ca27d None None] MFA Rules not processed for user `885c052c69ae48369fdae8647e8613b4`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:21:48.489715 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-29ee854a-e4f6-458d-83db-2501b45ca27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:48.557941 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1443/2901] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:21:48 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:21:48.569403 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1a27c-10e9-4920-b10b-bf4e9d226789 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:48.571388 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-86f1a27c-10e9-4920-b10b-bf4e9d226789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:21:48.628499 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1a27c-10e9-4920-b10b-bf4e9d226789 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:48.632197 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ded58-2627-4ec2-9e5b-5d760320ad3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:48.632793 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ded58-2627-4ec2-9e5b-5d760320ad3b service cinder] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:48.633580 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ded58-2627-4ec2-9e5b-5d760320ad3b service cinder] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:48.633924 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ded58-2627-4ec2-9e5b-5d760320ad3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:48.642854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ded58-2627-4ec2-9e5b-5d760320ad3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:48.644337 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ded58-2627-4ec2-9e5b-5d760320ad3b service cinder] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:48.654179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1459/2902] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:48 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:21:48.859140 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee854a-e4f6-458d-83db-2501b45ca27d None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:21:48.869094 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee854a-e4f6-458d-83db-2501b45ca27d None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:21:48.872784 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66092fc5-3981-4ad0-8838-ee035282b0cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:21:48.873382 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66092fc5-3981-4ad0-8838-ee035282b0cb service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:21:48.873659 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66092fc5-3981-4ad0-8838-ee035282b0cb service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:21:48.873962 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66092fc5-3981-4ad0-8838-ee035282b0cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:21:48.932309 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66092fc5-3981-4ad0-8838-ee035282b0cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:21:48.933724 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66092fc5-3981-4ad0-8838-ee035282b0cb service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:21:48.942719 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1444/2903] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:21:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:21:58.650024 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:21:58.938538 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=86 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:22:00.546256 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ded58-2627-4ec2-9e5b-5d760320ad3b service cinder] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:00.607086 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ded58-2627-4ec2-9e5b-5d760320ad3b service cinder] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:00.611610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07951078-37e7-4341-89fc-ebbdc615bf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:00.612251 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07951078-37e7-4341-89fc-ebbdc615bf7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:00.612524 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07951078-37e7-4341-89fc-ebbdc615bf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:00.612807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07951078-37e7-4341-89fc-ebbdc615bf7a admin admin] PATH_INFO: `/v3/users/13a21a526f31480f878a697f82c3c9de` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:00.613884 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07951078-37e7-4341-89fc-ebbdc615bf7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=13a21a526f31480f878a697f82c3c9de)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:00.615774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07951078-37e7-4341-89fc-ebbdc615bf7a admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:00.884481 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-07951078-37e7-4341-89fc-ebbdc615bf7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13a21a526f31480f878a697f82c3c9de'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:00.885148 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-07951078-37e7-4341-89fc-ebbdc615bf7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13a21a526f31480f878a697f82c3c9de'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:00.893076 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-07951078-37e7-4341-89fc-ebbdc615bf7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13a21a526f31480f878a697f82c3c9de'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:00.973656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-07951078-37e7-4341-89fc-ebbdc615bf7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13a21a526f31480f878a697f82c3c9de'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:00.974415 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-07951078-37e7-4341-89fc-ebbdc615bf7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13a21a526f31480f878a697f82c3c9de'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:00.993099 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1460/2904] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:22:00 2022] DELETE /identity/v3/users/13a21a526f31480f878a697f82c3c9de => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:22:00.997643 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aff564-3cc9-4258-91c6-c24ad5dafa8c None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:00.998113 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aff564-3cc9-4258-91c6-c24ad5dafa8c None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:00.998230 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aff564-3cc9-4258-91c6-c24ad5dafa8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:01.017770 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-a9aff564-3cc9-4258-91c6-c24ad5dafa8c None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:22:01.018955 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a9aff564-3cc9-4258-91c6-c24ad5dafa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:01.076679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1445/2905] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:22:01.086645 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07951078-37e7-4341-89fc-ebbdc615bf7a admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:01.088053 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-07951078-37e7-4341-89fc-ebbdc615bf7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:01.146583 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07951078-37e7-4341-89fc-ebbdc615bf7a admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:01.150145 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced06fae-55d1-40f8-b216-67b3b764f8fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:01.150744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced06fae-55d1-40f8-b216-67b3b764f8fc service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:01.151017 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced06fae-55d1-40f8-b216-67b3b764f8fc service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:01.151284 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced06fae-55d1-40f8-b216-67b3b764f8fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:01.160632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced06fae-55d1-40f8-b216-67b3b764f8fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:01.162043 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced06fae-55d1-40f8-b216-67b3b764f8fc service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:01.171457 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1461/2906] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:22:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:22:01.563731 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aff564-3cc9-4258-91c6-c24ad5dafa8c None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:01.565388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a9aff564-3cc9-4258-91c6-c24ad5dafa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:01.579541 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aff564-3cc9-4258-91c6-c24ad5dafa8c None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:01.584500 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c392a-a4a4-4c9b-ae1c-79a2e4f09fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:01.585137 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c392a-a4a4-4c9b-ae1c-79a2e4f09fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:01.585413 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c392a-a4a4-4c9b-ae1c-79a2e4f09fac admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:01.585682 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c392a-a4a4-4c9b-ae1c-79a2e4f09fac admin admin] PATH_INFO: `/v3/projects/c7b8e0ea82a14e1a86c3bc9610edd1bd` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:01.586821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c392a-a4a4-4c9b-ae1c-79a2e4f09fac admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7b8e0ea82a14e1a86c3bc9610edd1bd)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:01.588734 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c392a-a4a4-4c9b-ae1c-79a2e4f09fac admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:01.691643 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-580c392a-a4a4-4c9b-ae1c-79a2e4f09fac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7b8e0ea82a14e1a86c3bc9610edd1bd', 'request_id': 'req-580c392a-a4a4-4c9b-ae1c-79a2e4f09fac'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:01.761835 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-580c392a-a4a4-4c9b-ae1c-79a2e4f09fac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7b8e0ea82a14e1a86c3bc9610edd1bd', 'request_id': 'req-580c392a-a4a4-4c9b-ae1c-79a2e4f09fac'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:01.767647 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-580c392a-a4a4-4c9b-ae1c-79a2e4f09fac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7b8e0ea82a14e1a86c3bc9610edd1bd', 'request_id': 'req-580c392a-a4a4-4c9b-ae1c-79a2e4f09fac'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:01.778618 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-580c392a-a4a4-4c9b-ae1c-79a2e4f09fac admin admin] The token cache is being invalidate because project c7b8e0ea82a14e1a86c3bc9610edd1bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:22:01.778902 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-580c392a-a4a4-4c9b-ae1c-79a2e4f09fac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:01.779532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-580c392a-a4a4-4c9b-ae1c-79a2e4f09fac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:01.781023 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1446/2907] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:22:01 2022] DELETE /identity/v3/projects/c7b8e0ea82a14e1a86c3bc9610edd1bd => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:22:01.785835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced06fae-55d1-40f8-b216-67b3b764f8fc service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:01.795398 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced06fae-55d1-40f8-b216-67b3b764f8fc service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:01.813714 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:22:01.852522 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487295b-4fd6-458a-8828-bb3e3280923e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:01.853175 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487295b-4fd6-458a-8828-bb3e3280923e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:01.853455 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487295b-4fd6-458a-8828-bb3e3280923e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:01.853728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487295b-4fd6-458a-8828-bb3e3280923e admin admin] PATH_INFO: `/v3/users/885c052c69ae48369fdae8647e8613b4` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:01.854878 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7487295b-4fd6-458a-8828-bb3e3280923e admin admin] RBAC: Authorizing `identity:delete_user(user_id=885c052c69ae48369fdae8647e8613b4)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:01.856845 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7487295b-4fd6-458a-8828-bb3e3280923e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:02.027800 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7487295b-4fd6-458a-8828-bb3e3280923e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '885c052c69ae48369fdae8647e8613b4'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:02.028560 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7487295b-4fd6-458a-8828-bb3e3280923e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '885c052c69ae48369fdae8647e8613b4'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:02.036524 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7487295b-4fd6-458a-8828-bb3e3280923e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '885c052c69ae48369fdae8647e8613b4'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:02.105043 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7487295b-4fd6-458a-8828-bb3e3280923e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '885c052c69ae48369fdae8647e8613b4'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:02.105659 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7487295b-4fd6-458a-8828-bb3e3280923e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '885c052c69ae48369fdae8647e8613b4'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:02.123971 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1462/2908] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:22:01 2022] DELETE /identity/v3/users/885c052c69ae48369fdae8647e8613b4 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:22:02.560377 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c392a-a4a4-4c9b-ae1c-79a2e4f09fac admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:02.561897 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-580c392a-a4a4-4c9b-ae1c-79a2e4f09fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:02.577393 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c392a-a4a4-4c9b-ae1c-79a2e4f09fac admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:02.644035 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bc01d-77dd-4207-b2ac-4f6589d598b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:02.644729 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bc01d-77dd-4207-b2ac-4f6589d598b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:02.645005 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bc01d-77dd-4207-b2ac-4f6589d598b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:02.645281 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bc01d-77dd-4207-b2ac-4f6589d598b9 admin admin] PATH_INFO: `/v3/projects/0e5ac67c44da4bfd9c9c85f144e86b23` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:02.647375 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142bc01d-77dd-4207-b2ac-4f6589d598b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e5ac67c44da4bfd9c9c85f144e86b23)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:02.648359 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142bc01d-77dd-4207-b2ac-4f6589d598b9 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:02.769394 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:22:02.785242 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-142bc01d-77dd-4207-b2ac-4f6589d598b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e5ac67c44da4bfd9c9c85f144e86b23', 'request_id': 'req-142bc01d-77dd-4207-b2ac-4f6589d598b9'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:02.910855 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-142bc01d-77dd-4207-b2ac-4f6589d598b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e5ac67c44da4bfd9c9c85f144e86b23', 'request_id': 'req-142bc01d-77dd-4207-b2ac-4f6589d598b9'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:02.916799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-142bc01d-77dd-4207-b2ac-4f6589d598b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e5ac67c44da4bfd9c9c85f144e86b23', 'request_id': 'req-142bc01d-77dd-4207-b2ac-4f6589d598b9'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:02.934186 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-142bc01d-77dd-4207-b2ac-4f6589d598b9 admin admin] The token cache is being invalidate because project 0e5ac67c44da4bfd9c9c85f144e86b23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:22:02.934475 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-142bc01d-77dd-4207-b2ac-4f6589d598b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:02.935135 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-142bc01d-77dd-4207-b2ac-4f6589d598b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:02.936725 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1447/2909] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:22:02 2022] DELETE /identity/v3/projects/0e5ac67c44da4bfd9c9c85f144e86b23 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:22:02.942694 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd09b24-c7af-4c48-930d-a03d7b1f56d6 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:02.943167 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd09b24-c7af-4c48-930d-a03d7b1f56d6 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:02.943314 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd09b24-c7af-4c48-930d-a03d7b1f56d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:02.963632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-fdd09b24-c7af-4c48-930d-a03d7b1f56d6 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:22:02.964938 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-fdd09b24-c7af-4c48-930d-a03d7b1f56d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:03.024754 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1463/2910] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:22:03.040512 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625944de-418c-4ba0-897c-660883c086ba None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:03.040802 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625944de-418c-4ba0-897c-660883c086ba None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:03.041107 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625944de-418c-4ba0-897c-660883c086ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:03.062180 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-625944de-418c-4ba0-897c-660883c086ba None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:22:03.063415 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-625944de-418c-4ba0-897c-660883c086ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:03.071335 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1448/2911] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:22:03.075657 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd09b24-c7af-4c48-930d-a03d7b1f56d6 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:03.085428 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd09b24-c7af-4c48-930d-a03d7b1f56d6 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:03.089972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d5905-c0f2-4ff0-91fc-de713696015b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:03.090569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d5905-c0f2-4ff0-91fc-de713696015b admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:03.090841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d5905-c0f2-4ff0-91fc-de713696015b admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:03.091106 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d5905-c0f2-4ff0-91fc-de713696015b admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:03.091744 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d5905-c0f2-4ff0-91fc-de713696015b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:22:03.092040 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d5905-c0f2-4ff0-91fc-de713696015b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:03.093662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d5905-c0f2-4ff0-91fc-de713696015b admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:03.105768 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1464/2912] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:22:03.110142 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625944de-418c-4ba0-897c-660883c086ba None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:03.120161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625944de-418c-4ba0-897c-660883c086ba None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:03.124843 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b3851-828b-4c75-841c-c7572fc5bb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:03.125463 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b3851-828b-4c75-841c-c7572fc5bb67 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:03.125735 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b3851-828b-4c75-841c-c7572fc5bb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:03.126009 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b3851-828b-4c75-841c-c7572fc5bb67 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:03.126728 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b3851-828b-4c75-841c-c7572fc5bb67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:03.128827 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b3851-828b-4c75-841c-c7572fc5bb67 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:03.215112 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1449/2913] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:22:03 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:22:03.219200 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d5905-c0f2-4ff0-91fc-de713696015b admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:03.229667 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d5905-c0f2-4ff0-91fc-de713696015b admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:03.255272 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:22:03.284842 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508660d1-0a6c-4646-975c-3d147a26605e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:03.285732 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508660d1-0a6c-4646-975c-3d147a26605e admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:03.286007 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508660d1-0a6c-4646-975c-3d147a26605e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:03.286280 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508660d1-0a6c-4646-975c-3d147a26605e admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:03.286943 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508660d1-0a6c-4646-975c-3d147a26605e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:03.289728 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508660d1-0a6c-4646-975c-3d147a26605e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:03.370926 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1465/2914] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:22:03 2022] POST /identity/v3/users => generated 292 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:22:03.375168 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b3851-828b-4c75-841c-c7572fc5bb67 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:03.385666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b3851-828b-4c75-841c-c7572fc5bb67 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:03.390219 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105d4cb-e47c-4539-8806-1840a1fa17a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:03.390810 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105d4cb-e47c-4539-8806-1840a1fa17a6 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:03.391084 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105d4cb-e47c-4539-8806-1840a1fa17a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:03.391351 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105d4cb-e47c-4539-8806-1840a1fa17a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:03.392485 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105d4cb-e47c-4539-8806-1840a1fa17a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:03.394049 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105d4cb-e47c-4539-8806-1840a1fa17a6 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:03.403725 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1450/2915] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:22:03.407629 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508660d1-0a6c-4646-975c-3d147a26605e admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:03.418551 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508660d1-0a6c-4646-975c-3d147a26605e admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:03.423309 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d8770-f9d1-4b7d-8400-9fb34feac15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:03.423940 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d8770-f9d1-4b7d-8400-9fb34feac15f admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:03.424252 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d8770-f9d1-4b7d-8400-9fb34feac15f admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:03.424523 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d8770-f9d1-4b7d-8400-9fb34feac15f admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:03.425156 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d8770-f9d1-4b7d-8400-9fb34feac15f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:03.426733 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d8770-f9d1-4b7d-8400-9fb34feac15f admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:03.437311 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1466/2916] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:22:03.441203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105d4cb-e47c-4539-8806-1840a1fa17a6 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:03.451033 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105d4cb-e47c-4539-8806-1840a1fa17a6 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:03.458432 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8605e-99ec-40f8-b411-87ebd0dcae79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:03.459064 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8605e-99ec-40f8-b411-87ebd0dcae79 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:03.459336 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8605e-99ec-40f8-b411-87ebd0dcae79 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:03.459605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8605e-99ec-40f8-b411-87ebd0dcae79 admin admin] PATH_INFO: `/v3/projects/003a7e8da6c54826bd95cce1cb49035c/users/5e2f0fb034c042418234a07585495d1d/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:03.474554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8605e-99ec-40f8-b411-87ebd0dcae79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=003a7e8da6c54826bd95cce1cb49035c, user_id=5e2f0fb034c042418234a07585495d1d, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:03.477145 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8605e-99ec-40f8-b411-87ebd0dcae79 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:03.551084 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1451/2917] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:22:03 2022] PUT /identity/v3/projects/003a7e8da6c54826bd95cce1cb49035c/users/5e2f0fb034c042418234a07585495d1d/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:22:03.562555 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19243b6f-27db-45a1-8b0a-91329421c139 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:03.562923 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19243b6f-27db-45a1-8b0a-91329421c139 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:03.563192 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19243b6f-27db-45a1-8b0a-91329421c139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:03.591321 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-19243b6f-27db-45a1-8b0a-91329421c139 None None] MFA Rules not processed for user `5e2f0fb034c042418234a07585495d1d`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:22:03.592572 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-19243b6f-27db-45a1-8b0a-91329421c139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:03.660891 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1467/2918] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:22:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:22:03.673171 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8605e-99ec-40f8-b411-87ebd0dcae79 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:03.675012 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-e1d8605e-99ec-40f8-b411-87ebd0dcae79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:03.733704 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8605e-99ec-40f8-b411-87ebd0dcae79 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:03.737326 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f9898-91e4-47b0-ae1a-9906c4c8b5fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:03.737934 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f9898-91e4-47b0-ae1a-9906c4c8b5fd service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:03.738209 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f9898-91e4-47b0-ae1a-9906c4c8b5fd service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:03.738478 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f9898-91e4-47b0-ae1a-9906c4c8b5fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:03.746882 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f9898-91e4-47b0-ae1a-9906c4c8b5fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:03.748323 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f9898-91e4-47b0-ae1a-9906c4c8b5fd service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:03.757892 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1452/2919] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:22:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:22:03.779348 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19243b6f-27db-45a1-8b0a-91329421c139 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:03.841401 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19243b6f-27db-45a1-8b0a-91329421c139 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:03.846843 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4db6d-5385-4d37-bbd3-f1727e42113e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:03.847465 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4db6d-5385-4d37-bbd3-f1727e42113e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:03.847743 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4db6d-5385-4d37-bbd3-f1727e42113e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:03.848013 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4db6d-5385-4d37-bbd3-f1727e42113e admin admin] PATH_INFO: `/v3/users/5e2f0fb034c042418234a07585495d1d` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:03.849276 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e4db6d-5385-4d37-bbd3-f1727e42113e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e2f0fb034c042418234a07585495d1d)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:03.851181 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e4db6d-5385-4d37-bbd3-f1727e42113e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:04.048230 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:22:04.054807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-96e4db6d-5385-4d37-bbd3-f1727e42113e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e2f0fb034c042418234a07585495d1d'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:04.055663 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-96e4db6d-5385-4d37-bbd3-f1727e42113e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e2f0fb034c042418234a07585495d1d'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:04.062313 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-96e4db6d-5385-4d37-bbd3-f1727e42113e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e2f0fb034c042418234a07585495d1d'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:04.116511 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-96e4db6d-5385-4d37-bbd3-f1727e42113e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e2f0fb034c042418234a07585495d1d'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:04.117138 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-96e4db6d-5385-4d37-bbd3-f1727e42113e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e2f0fb034c042418234a07585495d1d'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:04.135668 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1468/2920] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:22:03 2022] DELETE /identity/v3/users/5e2f0fb034c042418234a07585495d1d => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:22:04.140117 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89bb73-43df-4384-81b3-2a1d0bf50845 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:04.140429 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89bb73-43df-4384-81b3-2a1d0bf50845 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:04.140726 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89bb73-43df-4384-81b3-2a1d0bf50845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:04.161170 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-fe89bb73-43df-4384-81b3-2a1d0bf50845 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:22:04.162388 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-fe89bb73-43df-4384-81b3-2a1d0bf50845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:04.209062 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:22:04.221405 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1453/2921] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:22:04.231157 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4db6d-5385-4d37-bbd3-f1727e42113e admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:04.232667 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-96e4db6d-5385-4d37-bbd3-f1727e42113e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:04.291559 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4db6d-5385-4d37-bbd3-f1727e42113e admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:04.295216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c85a48-387c-4abb-a378-7075afb0bec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:04.295813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c85a48-387c-4abb-a378-7075afb0bec1 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:04.296090 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c85a48-387c-4abb-a378-7075afb0bec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:04.296389 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c85a48-387c-4abb-a378-7075afb0bec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:04.305610 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c85a48-387c-4abb-a378-7075afb0bec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:04.306976 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c85a48-387c-4abb-a378-7075afb0bec1 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:04.316795 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1469/2922] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:22:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:22:04.718425 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89bb73-43df-4384-81b3-2a1d0bf50845 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:04.721020 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-fe89bb73-43df-4384-81b3-2a1d0bf50845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:04.735708 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89bb73-43df-4384-81b3-2a1d0bf50845 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:04.741783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f98c11-45cb-4666-8c7e-03b3f8fc92a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:04.742466 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f98c11-45cb-4666-8c7e-03b3f8fc92a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:04.742686 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f98c11-45cb-4666-8c7e-03b3f8fc92a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:04.742956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f98c11-45cb-4666-8c7e-03b3f8fc92a0 admin admin] PATH_INFO: `/v3/projects/003a7e8da6c54826bd95cce1cb49035c` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:04.744148 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f98c11-45cb-4666-8c7e-03b3f8fc92a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=003a7e8da6c54826bd95cce1cb49035c)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:04.749978 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f98c11-45cb-4666-8c7e-03b3f8fc92a0 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:04.863874 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-96f98c11-45cb-4666-8c7e-03b3f8fc92a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '003a7e8da6c54826bd95cce1cb49035c', 'request_id': 'req-96f98c11-45cb-4666-8c7e-03b3f8fc92a0'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:04.927994 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-96f98c11-45cb-4666-8c7e-03b3f8fc92a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '003a7e8da6c54826bd95cce1cb49035c', 'request_id': 'req-96f98c11-45cb-4666-8c7e-03b3f8fc92a0'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:04.937169 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-96f98c11-45cb-4666-8c7e-03b3f8fc92a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '003a7e8da6c54826bd95cce1cb49035c', 'request_id': 'req-96f98c11-45cb-4666-8c7e-03b3f8fc92a0'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:04.950335 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-96f98c11-45cb-4666-8c7e-03b3f8fc92a0 admin admin] The token cache is being invalidate because project 003a7e8da6c54826bd95cce1cb49035c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:22:04.950624 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-96f98c11-45cb-4666-8c7e-03b3f8fc92a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:04.951371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-96f98c11-45cb-4666-8c7e-03b3f8fc92a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:04.952890 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1454/2923] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:22:04 2022] DELETE /identity/v3/projects/003a7e8da6c54826bd95cce1cb49035c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:22:04.958858 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df23743-96f8-4828-a2a0-485c87268413 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:04.959158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df23743-96f8-4828-a2a0-485c87268413 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:04.959437 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df23743-96f8-4828-a2a0-485c87268413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:04.982388 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-4df23743-96f8-4828-a2a0-485c87268413 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:22:04.983715 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-4df23743-96f8-4828-a2a0-485c87268413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:05.049205 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1470/2924] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:22:05.056981 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744deeb-464c-4d02-8379-a691e83485c0 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:05.057264 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744deeb-464c-4d02-8379-a691e83485c0 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:05.057532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744deeb-464c-4d02-8379-a691e83485c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:05.078291 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-2744deeb-464c-4d02-8379-a691e83485c0 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:22:05.080018 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-2744deeb-464c-4d02-8379-a691e83485c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:05.088780 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1455/2925] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:22:05.093242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df23743-96f8-4828-a2a0-485c87268413 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:05.103429 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df23743-96f8-4828-a2a0-485c87268413 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:05.108021 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a8613-b1df-491b-beb8-1a7d24bbeb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:05.108657 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a8613-b1df-491b-beb8-1a7d24bbeb14 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:05.108935 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a8613-b1df-491b-beb8-1a7d24bbeb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:05.109199 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a8613-b1df-491b-beb8-1a7d24bbeb14 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:05.109836 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a8613-b1df-491b-beb8-1a7d24bbeb14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:22:05.110131 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a8613-b1df-491b-beb8-1a7d24bbeb14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:05.112073 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a8613-b1df-491b-beb8-1a7d24bbeb14 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:05.128032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1471/2926] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 23:22:05.131075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744deeb-464c-4d02-8379-a691e83485c0 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:05.140543 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744deeb-464c-4d02-8379-a691e83485c0 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:05.145806 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14319b-b664-4544-a9fa-6c9b95a1be07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:05.146402 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14319b-b664-4544-a9fa-6c9b95a1be07 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:05.146679 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14319b-b664-4544-a9fa-6c9b95a1be07 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:05.146946 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14319b-b664-4544-a9fa-6c9b95a1be07 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:05.147609 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14319b-b664-4544-a9fa-6c9b95a1be07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:05.149530 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14319b-b664-4544-a9fa-6c9b95a1be07 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:05.232116 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1456/2927] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:22:05 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:22:05.236598 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a8613-b1df-491b-beb8-1a7d24bbeb14 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:05.246540 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a8613-b1df-491b-beb8-1a7d24bbeb14 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:05.291554 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:22:05.303997 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a3126-7be3-4bab-a578-c989ab6d4f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:05.314521 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a3126-7be3-4bab-a578-c989ab6d4f10 admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:05.314521 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a3126-7be3-4bab-a578-c989ab6d4f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:05.314521 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a3126-7be3-4bab-a578-c989ab6d4f10 admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:05.314521 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a3126-7be3-4bab-a578-c989ab6d4f10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:05.314521 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a3126-7be3-4bab-a578-c989ab6d4f10 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:05.451662 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1472/2928] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:22:05 2022] POST /identity/v3/users => generated 286 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:22:05.457141 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14319b-b664-4544-a9fa-6c9b95a1be07 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:05.467308 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14319b-b664-4544-a9fa-6c9b95a1be07 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:05.471974 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e27dbc-93dc-456b-b876-84252149f59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:05.472602 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e27dbc-93dc-456b-b876-84252149f59f admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:05.472938 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e27dbc-93dc-456b-b876-84252149f59f admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:05.473214 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e27dbc-93dc-456b-b876-84252149f59f admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:05.473849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e27dbc-93dc-456b-b876-84252149f59f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:05.475451 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e27dbc-93dc-456b-b876-84252149f59f admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:05.485215 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1457/2929] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:22:05.489071 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a3126-7be3-4bab-a578-c989ab6d4f10 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:05.498767 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a3126-7be3-4bab-a578-c989ab6d4f10 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:05.504395 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1b6f0-1948-4798-84a7-028adc490848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:05.507290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1b6f0-1948-4798-84a7-028adc490848 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:05.507290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1b6f0-1948-4798-84a7-028adc490848 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:05.507290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1b6f0-1948-4798-84a7-028adc490848 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:05.507290 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1b6f0-1948-4798-84a7-028adc490848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:05.507862 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1b6f0-1948-4798-84a7-028adc490848 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:05.520285 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1473/2930] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:22:05.524319 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e27dbc-93dc-456b-b876-84252149f59f admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:05.534250 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e27dbc-93dc-456b-b876-84252149f59f admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:05.539129 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef569cdb-b8ef-4255-bf69-3dc8aa95bac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:05.539773 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef569cdb-b8ef-4255-bf69-3dc8aa95bac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:05.541720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef569cdb-b8ef-4255-bf69-3dc8aa95bac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:05.541720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef569cdb-b8ef-4255-bf69-3dc8aa95bac9 admin admin] PATH_INFO: `/v3/projects/c4d8d7ba70a045aca8ef8982ca1302b0/users/8df15176597f46149bc4ffae0463c12a/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:05.555078 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef569cdb-b8ef-4255-bf69-3dc8aa95bac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d8d7ba70a045aca8ef8982ca1302b0, user_id=8df15176597f46149bc4ffae0463c12a, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:05.557666 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef569cdb-b8ef-4255-bf69-3dc8aa95bac9 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:05.674271 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1458/2931] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:22:05 2022] PUT /identity/v3/projects/c4d8d7ba70a045aca8ef8982ca1302b0/users/8df15176597f46149bc4ffae0463c12a/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:22:05.679519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac89574-4c1e-4f78-9425-2462e36d380c None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:05.679797 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac89574-4c1e-4f78-9425-2462e36d380c None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:05.680064 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac89574-4c1e-4f78-9425-2462e36d380c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:05.699823 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-3ac89574-4c1e-4f78-9425-2462e36d380c None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:22:05.701033 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-3ac89574-4c1e-4f78-9425-2462e36d380c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:05.758543 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1474/2932] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:22:05.768519 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef569cdb-b8ef-4255-bf69-3dc8aa95bac9 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:05.770250 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ef569cdb-b8ef-4255-bf69-3dc8aa95bac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:05.828492 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef569cdb-b8ef-4255-bf69-3dc8aa95bac9 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:05.832857 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3080d5-3cd0-4e75-bb53-b003ec8d403e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:05.833452 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3080d5-3cd0-4e75-bb53-b003ec8d403e service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:05.833725 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3080d5-3cd0-4e75-bb53-b003ec8d403e service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:05.833992 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3080d5-3cd0-4e75-bb53-b003ec8d403e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:05.842821 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3080d5-3cd0-4e75-bb53-b003ec8d403e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:05.844261 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3080d5-3cd0-4e75-bb53-b003ec8d403e service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:05.855456 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1459/2933] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:22:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:22:07.365854 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11428d7a-3ac0-4192-a854-4deea76d60ec None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:07.366363 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11428d7a-3ac0-4192-a854-4deea76d60ec None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:07.366450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11428d7a-3ac0-4192-a854-4deea76d60ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:07.416778 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-11428d7a-3ac0-4192-a854-4deea76d60ec None None] MFA Rules not processed for user `8df15176597f46149bc4ffae0463c12a`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:22:07.418006 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-11428d7a-3ac0-4192-a854-4deea76d60ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:07.491007 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1475/2934] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:22:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:22:07.503742 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3080d5-3cd0-4e75-bb53-b003ec8d403e service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:07.505449 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-9e3080d5-3cd0-4e75-bb53-b003ec8d403e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:07.524711 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:22:07.563028 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3080d5-3cd0-4e75-bb53-b003ec8d403e service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:07.567522 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f2d40-ba1b-411b-a982-fdf54e4777ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:07.568350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f2d40-ba1b-411b-a982-fdf54e4777ee service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:07.568853 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f2d40-ba1b-411b-a982-fdf54e4777ee service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:07.569173 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f2d40-ba1b-411b-a982-fdf54e4777ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:07.578312 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f2d40-ba1b-411b-a982-fdf54e4777ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:07.580592 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f2d40-ba1b-411b-a982-fdf54e4777ee service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:07.589994 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1460/2935] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:22:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:22:09.319954 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba68dda-0855-4fd8-b7c6-63d081c45610 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:09.321343 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba68dda-0855-4fd8-b7c6-63d081c45610 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:09.321343 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba68dda-0855-4fd8-b7c6-63d081c45610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:09.340032 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-bba68dda-0855-4fd8-b7c6-63d081c45610 None None] MFA Rules not processed for user `8df15176597f46149bc4ffae0463c12a`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:22:09.341162 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-bba68dda-0855-4fd8-b7c6-63d081c45610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:09.347975 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1476/2936] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:22:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:22:09.577316 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f2d40-ba1b-411b-a982-fdf54e4777ee service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:09.586697 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f2d40-ba1b-411b-a982-fdf54e4777ee service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:09.590344 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd22463-fec6-4bc8-8066-613e928c0a53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:09.590873 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd22463-fec6-4bc8-8066-613e928c0a53 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:09.591150 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd22463-fec6-4bc8-8066-613e928c0a53 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:09.591419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd22463-fec6-4bc8-8066-613e928c0a53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:09.593148 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-4cd22463-fec6-4bc8-8066-613e928c0a53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:09.602598 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd22463-fec6-4bc8-8066-613e928c0a53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:09.604003 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd22463-fec6-4bc8-8066-613e928c0a53 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:09.611908 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1461/2937] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:22:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:22:11.219887 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba68dda-0855-4fd8-b7c6-63d081c45610 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:11.222123 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-bba68dda-0855-4fd8-b7c6-63d081c45610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:11.234243 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba68dda-0855-4fd8-b7c6-63d081c45610 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:11.237846 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a5e20-61ed-4135-8b24-cebae51d7494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:11.238450 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a5e20-61ed-4135-8b24-cebae51d7494 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:11.238743 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a5e20-61ed-4135-8b24-cebae51d7494 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:11.239007 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a5e20-61ed-4135-8b24-cebae51d7494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:11.240729 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-361a5e20-61ed-4135-8b24-cebae51d7494 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:11.286270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:22:11.303612 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a5e20-61ed-4135-8b24-cebae51d7494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:11.305243 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a5e20-61ed-4135-8b24-cebae51d7494 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:11.314951 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1477/2938] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:22:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:22:19.608427 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=25 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:22:21.470843 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=8 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:22:29.222193 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9bce0-a9f5-413b-b764-a6b8174e6541 None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:29.222654 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9bce0-a9f5-413b-b764-a6b8174e6541 None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:29.222741 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9bce0-a9f5-413b-b764-a6b8174e6541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:29.242214 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-3fa9bce0-a9f5-413b-b764-a6b8174e6541 None None] MFA Rules not processed for user `8df15176597f46149bc4ffae0463c12a`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:22:29.244108 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3fa9bce0-a9f5-413b-b764-a6b8174e6541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:29.250988 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1462/2939] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:22:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:22:37.085465 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfacf9-d112-40c6-a42c-96dbc3a35922 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:37.085919 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfacf9-d112-40c6-a42c-96dbc3a35922 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:37.087901 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfacf9-d112-40c6-a42c-96dbc3a35922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:37.105721 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-06cfacf9-d112-40c6-a42c-96dbc3a35922 None None] MFA Rules not processed for user `980157efae8f43b59dc063fda04c0348`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:22:37.106910 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-06cfacf9-d112-40c6-a42c-96dbc3a35922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:37.164476 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1478/2940] 10.222.0.59 () {58 vars in 967 bytes} [Mon Aug 29 23:22:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:22:37.307446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9bce0-a9f5-413b-b764-a6b8174e6541 None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:37.307446 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3fa9bce0-a9f5-413b-b764-a6b8174e6541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:37.380964 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9bce0-a9f5-413b-b764-a6b8174e6541 None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:37.386049 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36811918-6854-4dd6-938b-a3dd77c2c675 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f0503ae1b24ce6a8adb3cfd7973f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:37.386653 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36811918-6854-4dd6-938b-a3dd77c2c675 service swift] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:37.386930 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36811918-6854-4dd6-938b-a3dd77c2c675 service swift] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:37.387204 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36811918-6854-4dd6-938b-a3dd77c2c675 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:37.395568 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36811918-6854-4dd6-938b-a3dd77c2c675 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:37.397307 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36811918-6854-4dd6-938b-a3dd77c2c675 service swift] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:37.405412 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1463/2941] 10.222.0.59 () {60 vars in 1384 bytes} [Mon Aug 29 23:22:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 23:22:46.219255 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfacf9-d112-40c6-a42c-96dbc3a35922 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:46.221353 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-06cfacf9-d112-40c6-a42c-96dbc3a35922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:46.235841 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfacf9-d112-40c6-a42c-96dbc3a35922 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:46.240757 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d52fc-0b0e-4d40-b7c5-d3900126a21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:46.241383 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d52fc-0b0e-4d40-b7c5-d3900126a21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:46.241658 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d52fc-0b0e-4d40-b7c5-d3900126a21e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:46.241925 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d52fc-0b0e-4d40-b7c5-d3900126a21e admin admin] PATH_INFO: `/v3/users/79a2748b61b4451f9215a4f8eea5dea2` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:46.243111 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d52fc-0b0e-4d40-b7c5-d3900126a21e admin admin] RBAC: Authorizing `identity:delete_user(user_id=79a2748b61b4451f9215a4f8eea5dea2)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:46.245634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d52fc-0b0e-4d40-b7c5-d3900126a21e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:46.391063 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7b2d52fc-0b0e-4d40-b7c5-d3900126a21e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79a2748b61b4451f9215a4f8eea5dea2'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:46.391720 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7b2d52fc-0b0e-4d40-b7c5-d3900126a21e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79a2748b61b4451f9215a4f8eea5dea2'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:46.398548 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7b2d52fc-0b0e-4d40-b7c5-d3900126a21e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79a2748b61b4451f9215a4f8eea5dea2'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:46.463454 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7b2d52fc-0b0e-4d40-b7c5-d3900126a21e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79a2748b61b4451f9215a4f8eea5dea2'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:46.464101 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7b2d52fc-0b0e-4d40-b7c5-d3900126a21e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79a2748b61b4451f9215a4f8eea5dea2'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:46.482937 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1479/2942] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:22:46 2022] DELETE /identity/v3/users/79a2748b61b4451f9215a4f8eea5dea2 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:22:46.745974 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36811918-6854-4dd6-938b-a3dd77c2c675 service swift] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:46.757224 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36811918-6854-4dd6-938b-a3dd77c2c675 service swift] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:46.818052 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fa32f-c231-483f-8b9c-946e08c064e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:46.818682 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fa32f-c231-483f-8b9c-946e08c064e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:46.818956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fa32f-c231-483f-8b9c-946e08c064e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:46.819243 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fa32f-c231-483f-8b9c-946e08c064e7 admin admin] PATH_INFO: `/v3/projects/8b61dd30bd20426990ed70288f69e40b` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:46.820371 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421fa32f-c231-483f-8b9c-946e08c064e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b61dd30bd20426990ed70288f69e40b)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:46.822290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421fa32f-c231-483f-8b9c-946e08c064e7 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:46.952077 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-421fa32f-c231-483f-8b9c-946e08c064e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b61dd30bd20426990ed70288f69e40b', 'request_id': 'req-421fa32f-c231-483f-8b9c-946e08c064e7'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:47.018080 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-421fa32f-c231-483f-8b9c-946e08c064e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b61dd30bd20426990ed70288f69e40b', 'request_id': 'req-421fa32f-c231-483f-8b9c-946e08c064e7'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:47.024829 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-421fa32f-c231-483f-8b9c-946e08c064e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b61dd30bd20426990ed70288f69e40b', 'request_id': 'req-421fa32f-c231-483f-8b9c-946e08c064e7'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:47.037568 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-421fa32f-c231-483f-8b9c-946e08c064e7 admin admin] The token cache is being invalidate because project 8b61dd30bd20426990ed70288f69e40b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:22:47.037858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-421fa32f-c231-483f-8b9c-946e08c064e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:47.038503 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-421fa32f-c231-483f-8b9c-946e08c064e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:47.039968 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1464/2943] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:22:46 2022] DELETE /identity/v3/projects/8b61dd30bd20426990ed70288f69e40b => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:22:47.044811 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d52fc-0b0e-4d40-b7c5-d3900126a21e admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:47.046270 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-7b2d52fc-0b0e-4d40-b7c5-d3900126a21e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:47.059774 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d52fc-0b0e-4d40-b7c5-d3900126a21e admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:47.122754 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf71a1-87cd-4a9e-a1e8-613b0f84b22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:47.123301 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf71a1-87cd-4a9e-a1e8-613b0f84b22b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:47.123693 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf71a1-87cd-4a9e-a1e8-613b0f84b22b admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:47.123848 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf71a1-87cd-4a9e-a1e8-613b0f84b22b admin admin] PATH_INFO: `/v3/users/fc466057f33f409a9d5aa5dfbee8269c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:47.125142 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf71a1-87cd-4a9e-a1e8-613b0f84b22b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc466057f33f409a9d5aa5dfbee8269c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:47.127148 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf71a1-87cd-4a9e-a1e8-613b0f84b22b admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:47.150056 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:22:47.306365 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-23bf71a1-87cd-4a9e-a1e8-613b0f84b22b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc466057f33f409a9d5aa5dfbee8269c'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:47.307080 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-23bf71a1-87cd-4a9e-a1e8-613b0f84b22b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc466057f33f409a9d5aa5dfbee8269c'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:47.314216 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-23bf71a1-87cd-4a9e-a1e8-613b0f84b22b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc466057f33f409a9d5aa5dfbee8269c'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:47.389569 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-23bf71a1-87cd-4a9e-a1e8-613b0f84b22b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc466057f33f409a9d5aa5dfbee8269c'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:47.390307 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-23bf71a1-87cd-4a9e-a1e8-613b0f84b22b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc466057f33f409a9d5aa5dfbee8269c'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:47.408884 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1480/2944] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:22:47 2022] DELETE /identity/v3/users/fc466057f33f409a9d5aa5dfbee8269c => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:22:47.689231 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fa32f-c231-483f-8b9c-946e08c064e7 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:47.690659 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-421fa32f-c231-483f-8b9c-946e08c064e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:47.704613 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fa32f-c231-483f-8b9c-946e08c064e7 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:47.719782 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:22:47.768998 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae04e1-79dd-4c5a-b08b-2710187f5cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:47.769671 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae04e1-79dd-4c5a-b08b-2710187f5cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:47.769791 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae04e1-79dd-4c5a-b08b-2710187f5cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:47.770055 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae04e1-79dd-4c5a-b08b-2710187f5cc0 admin admin] PATH_INFO: `/v3/projects/321e257d63514d289308ea1b6b257528` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:47.771273 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae04e1-79dd-4c5a-b08b-2710187f5cc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=321e257d63514d289308ea1b6b257528)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:47.773191 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae04e1-79dd-4c5a-b08b-2710187f5cc0 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:47.899274 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-25ae04e1-79dd-4c5a-b08b-2710187f5cc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '321e257d63514d289308ea1b6b257528', 'request_id': 'req-25ae04e1-79dd-4c5a-b08b-2710187f5cc0'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:47.966783 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-25ae04e1-79dd-4c5a-b08b-2710187f5cc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '321e257d63514d289308ea1b6b257528', 'request_id': 'req-25ae04e1-79dd-4c5a-b08b-2710187f5cc0'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:47.972771 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-25ae04e1-79dd-4c5a-b08b-2710187f5cc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '321e257d63514d289308ea1b6b257528', 'request_id': 'req-25ae04e1-79dd-4c5a-b08b-2710187f5cc0'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:47.985147 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-25ae04e1-79dd-4c5a-b08b-2710187f5cc0 admin admin] The token cache is being invalidate because project 321e257d63514d289308ea1b6b257528 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:22:47.985443 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-25ae04e1-79dd-4c5a-b08b-2710187f5cc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:47.986097 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-25ae04e1-79dd-4c5a-b08b-2710187f5cc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:22:47.987605 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1465/2945] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:22:47 2022] DELETE /identity/v3/projects/321e257d63514d289308ea1b6b257528 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:22:54.803874 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db43fa-fbd3-4ae5-8b51-edf3454b337d None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:54.804339 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db43fa-fbd3-4ae5-8b51-edf3454b337d None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:54.804445 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db43fa-fbd3-4ae5-8b51-edf3454b337d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:54.827207 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-f1db43fa-fbd3-4ae5-8b51-edf3454b337d None None] MFA Rules not processed for user `980157efae8f43b59dc063fda04c0348`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:22:54.828463 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-f1db43fa-fbd3-4ae5-8b51-edf3454b337d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:54.884783 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1481/2946] 10.222.0.59 () {58 vars in 1009 bytes} [Mon Aug 29 23:22:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:22:54.896970 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae04e1-79dd-4c5a-b08b-2710187f5cc0 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:22:54.898343 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-25ae04e1-79dd-4c5a-b08b-2710187f5cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:22:54.969886 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae04e1-79dd-4c5a-b08b-2710187f5cc0 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:22:54.974910 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434cd2c-d60a-4a8d-b0e1-abceb70a4a03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f0503ae1b24ce6a8adb3cfd7973f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:22:54.975520 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434cd2c-d60a-4a8d-b0e1-abceb70a4a03 service swift] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:22:54.975796 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434cd2c-d60a-4a8d-b0e1-abceb70a4a03 service swift] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:22:54.976067 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434cd2c-d60a-4a8d-b0e1-abceb70a4a03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:22:54.984707 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c434cd2c-d60a-4a8d-b0e1-abceb70a4a03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:22:54.986279 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c434cd2c-d60a-4a8d-b0e1-abceb70a4a03 service swift] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:22:54.994195 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1466/2947] 10.222.0.59 () {60 vars in 1384 bytes} [Mon Aug 29 23:22:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 23:23:04.878015 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:DELETE=8,keystone:SELECT=45,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:23:04.991093 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:23:12.082632 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3e222-3d75-4afe-9795-cfe5cde991cd None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:23:12.083141 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3e222-3d75-4afe-9795-cfe5cde991cd None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:23:12.083244 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3e222-3d75-4afe-9795-cfe5cde991cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:23:12.102331 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-acf3e222-3d75-4afe-9795-cfe5cde991cd None None] MFA Rules not processed for user `980157efae8f43b59dc063fda04c0348`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:23:12.103533 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-acf3e222-3d75-4afe-9795-cfe5cde991cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:23:12.110693 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1482/2948] 10.222.0.59 () {58 vars in 967 bytes} [Mon Aug 29 23:23:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 23:23:12.127565 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434cd2c-d60a-4a8d-b0e1-abceb70a4a03 service swift] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:23:12.138544 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434cd2c-d60a-4a8d-b0e1-abceb70a4a03 service swift] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:23:12.143763 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5352f-c860-4ee9-990c-0810eac42193 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1f0503ae1b24ce6a8adb3cfd7973f14', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:23:12.144372 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5352f-c860-4ee9-990c-0810eac42193 service swift] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:23:12.144643 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5352f-c860-4ee9-990c-0810eac42193 service swift] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:23:12.144912 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5352f-c860-4ee9-990c-0810eac42193 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:23:12.153161 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5352f-c860-4ee9-990c-0810eac42193 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:23:12.155290 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5352f-c860-4ee9-990c-0810eac42193 service swift] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:23:12.163383 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1467/2949] 10.222.0.59 () {60 vars in 1384 bytes} [Mon Aug 29 23:23:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 23:23:14.983137 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3e222-3d75-4afe-9795-cfe5cde991cd None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:23:14.984627 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-acf3e222-3d75-4afe-9795-cfe5cde991cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:23:15.050813 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3e222-3d75-4afe-9795-cfe5cde991cd None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:23:15.055337 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e1ccd-f8ee-48cc-9e84-bf96b2a43816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:23:15.055956 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e1ccd-f8ee-48cc-9e84-bf96b2a43816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:23:15.056380 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e1ccd-f8ee-48cc-9e84-bf96b2a43816 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:23:15.056532 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e1ccd-f8ee-48cc-9e84-bf96b2a43816 admin admin] PATH_INFO: `/v3/users/8df15176597f46149bc4ffae0463c12a` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:23:15.057674 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e1ccd-f8ee-48cc-9e84-bf96b2a43816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8df15176597f46149bc4ffae0463c12a)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:23:15.059601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e1ccd-f8ee-48cc-9e84-bf96b2a43816 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:23:15.241189 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-501e1ccd-f8ee-48cc-9e84-bf96b2a43816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8df15176597f46149bc4ffae0463c12a'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:23:15.241845 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-501e1ccd-f8ee-48cc-9e84-bf96b2a43816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8df15176597f46149bc4ffae0463c12a'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:23:15.249472 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-501e1ccd-f8ee-48cc-9e84-bf96b2a43816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8df15176597f46149bc4ffae0463c12a'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:23:15.324518 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-501e1ccd-f8ee-48cc-9e84-bf96b2a43816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8df15176597f46149bc4ffae0463c12a'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:23:15.325161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-501e1ccd-f8ee-48cc-9e84-bf96b2a43816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8df15176597f46149bc4ffae0463c12a'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:23:15.344996 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1483/2950] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:23:14 2022] DELETE /identity/v3/users/8df15176597f46149bc4ffae0463c12a => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:23:15.577709 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5352f-c860-4ee9-990c-0810eac42193 service swift] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:23:15.590415 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5352f-c860-4ee9-990c-0810eac42193 service swift] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:23:15.654386 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c1262-32ed-4957-97be-58e449f0a121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:23:15.654386 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c1262-32ed-4957-97be-58e449f0a121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:23:15.654386 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c1262-32ed-4957-97be-58e449f0a121 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:23:15.654386 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c1262-32ed-4957-97be-58e449f0a121 admin admin] PATH_INFO: `/v3/projects/c4d8d7ba70a045aca8ef8982ca1302b0` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:23:15.654959 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c1262-32ed-4957-97be-58e449f0a121 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4d8d7ba70a045aca8ef8982ca1302b0)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:23:15.654959 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c1262-32ed-4957-97be-58e449f0a121 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:23:15.805834 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-ec0c1262-32ed-4957-97be-58e449f0a121 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4d8d7ba70a045aca8ef8982ca1302b0', 'request_id': 'req-ec0c1262-32ed-4957-97be-58e449f0a121'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:23:15.875720 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-ec0c1262-32ed-4957-97be-58e449f0a121 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4d8d7ba70a045aca8ef8982ca1302b0', 'request_id': 'req-ec0c1262-32ed-4957-97be-58e449f0a121'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:23:15.881665 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-ec0c1262-32ed-4957-97be-58e449f0a121 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4d8d7ba70a045aca8ef8982ca1302b0', 'request_id': 'req-ec0c1262-32ed-4957-97be-58e449f0a121'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:23:15.892594 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-ec0c1262-32ed-4957-97be-58e449f0a121 admin admin] The token cache is being invalidate because project c4d8d7ba70a045aca8ef8982ca1302b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:23:15.892875 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-ec0c1262-32ed-4957-97be-58e449f0a121 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:23:15.893498 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-ec0c1262-32ed-4957-97be-58e449f0a121 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:23:15.894963 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1468/2951] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:23:15 2022] DELETE /identity/v3/projects/c4d8d7ba70a045aca8ef8982ca1302b0 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:23:15.900120 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b1e66-2d4c-444d-9c24-c223ce81e681 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:23:15.900564 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b1e66-2d4c-444d-9c24-c223ce81e681 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:23:15.904695 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b1e66-2d4c-444d-9c24-c223ce81e681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:23:15.924815 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-fe0b1e66-2d4c-444d-9c24-c223ce81e681 None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:23:15.926345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-fe0b1e66-2d4c-444d-9c24-c223ce81e681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:23:15.986363 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1484/2952] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:23:15.994350 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26129eb-ff9b-40cb-864c-54c8dba7b8fe None None] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:23:15.994635 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26129eb-ff9b-40cb-864c-54c8dba7b8fe None None] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:23:15.994910 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26129eb-ff9b-40cb-864c-54c8dba7b8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:23:16.014874 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.auth.core [None req-f26129eb-ff9b-40cb-864c-54c8dba7b8fe None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:23:16.016165 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-f26129eb-ff9b-40cb-864c-54c8dba7b8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:23:16.024373 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1469/2953] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:23:16.028601 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b1e66-2d4c-444d-9c24-c223ce81e681 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:23:16.039608 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b1e66-2d4c-444d-9c24-c223ce81e681 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:23:16.044243 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb489622-6887-498d-9800-b1e555730be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:23:16.044864 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb489622-6887-498d-9800-b1e555730be3 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:23:16.045141 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb489622-6887-498d-9800-b1e555730be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:23:16.045418 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb489622-6887-498d-9800-b1e555730be3 admin admin] PATH_INFO: `/v3/domains` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:23:16.046055 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb489622-6887-498d-9800-b1e555730be3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 23:23:16.046356 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb489622-6887-498d-9800-b1e555730be3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:23:16.048684 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb489622-6887-498d-9800-b1e555730be3 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:23:16.057158 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:23:16.060696 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1485/2954] 10.222.0.59 () {58 vars in 1153 bytes} [Mon Aug 29 23:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 23:23:16.064647 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26129eb-ff9b-40cb-864c-54c8dba7b8fe None None] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:23:16.073913 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26129eb-ff9b-40cb-864c-54c8dba7b8fe None None] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:23:16.078811 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce1278-5b71-44fa-8d70-f5cf3b9c0873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:23:16.079403 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce1278-5b71-44fa-8d70-f5cf3b9c0873 admin admin] REQUEST_METHOD: `POST` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:23:16.079680 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce1278-5b71-44fa-8d70-f5cf3b9c0873 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:23:16.079945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce1278-5b71-44fa-8d70-f5cf3b9c0873 admin admin] PATH_INFO: `/v3/projects` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:23:16.080688 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce1278-5b71-44fa-8d70-f5cf3b9c0873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:23:16.082554 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce1278-5b71-44fa-8d70-f5cf3b9c0873 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:23:16.175101 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1470/2955] 10.222.0.59 () {60 vars in 1153 bytes} [Mon Aug 29 23:23:16 2022] POST /identity/v3/projects => generated 391 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:23:16.179143 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb489622-6887-498d-9800-b1e555730be3 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:23:16.189442 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb489622-6887-498d-9800-b1e555730be3 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:23:16.246704 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958d67e-bc0a-48a9-ac73-7e3ca181e08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:23:16.247299 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958d67e-bc0a-48a9-ac73-7e3ca181e08b admin admin] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:23:16.247579 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958d67e-bc0a-48a9-ac73-7e3ca181e08b admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:23:16.247847 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958d67e-bc0a-48a9-ac73-7e3ca181e08b admin admin] PATH_INFO: `/v3/users` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:23:16.248499 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958d67e-bc0a-48a9-ac73-7e3ca181e08b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:23:16.250599 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958d67e-bc0a-48a9-ac73-7e3ca181e08b admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:23:16.321844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1486/2956] 10.222.0.59 () {60 vars in 1144 bytes} [Mon Aug 29 23:23:16 2022] POST /identity/v3/users => generated 296 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 23:23:16.326098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce1278-5b71-44fa-8d70-f5cf3b9c0873 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:23:16.338419 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce1278-5b71-44fa-8d70-f5cf3b9c0873 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:23:16.343098 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15257229-5e99-456e-9a8b-7ed6178b1684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:23:16.344571 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15257229-5e99-456e-9a8b-7ed6178b1684 admin admin] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:23:16.344858 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15257229-5e99-456e-9a8b-7ed6178b1684 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:23:16.345148 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15257229-5e99-456e-9a8b-7ed6178b1684 admin admin] PATH_INFO: `/v3/roles` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:23:16.345784 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15257229-5e99-456e-9a8b-7ed6178b1684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:23:16.347368 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15257229-5e99-456e-9a8b-7ed6178b1684 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:23:16.357387 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1471/2957] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 23:23:16.361239 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958d67e-bc0a-48a9-ac73-7e3ca181e08b admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:23:16.371438 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958d67e-bc0a-48a9-ac73-7e3ca181e08b admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:23:16.376109 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8203c2e-8562-4fb3-af72-5a05123740f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:23:16.376893 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8203c2e-8562-4fb3-af72-5a05123740f2 admin admin] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:23:16.377170 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8203c2e-8562-4fb3-af72-5a05123740f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:23:16.377492 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8203c2e-8562-4fb3-af72-5a05123740f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:23:16.378125 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8203c2e-8562-4fb3-af72-5a05123740f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:23:16.379704 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8203c2e-8562-4fb3-af72-5a05123740f2 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:23:16.391070 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1487/2958] 10.222.0.59 () {58 vars in 1122 bytes} [Mon Aug 29 23:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 23:23:16.395103 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15257229-5e99-456e-9a8b-7ed6178b1684 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:23:16.405415 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15257229-5e99-456e-9a8b-7ed6178b1684 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:23:16.410809 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f9d7d-68a5-4506-9b00-14513999250d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:23:16.411527 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f9d7d-68a5-4506-9b00-14513999250d admin admin] REQUEST_METHOD: `PUT` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:23:16.411809 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f9d7d-68a5-4506-9b00-14513999250d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:23:16.412391 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f9d7d-68a5-4506-9b00-14513999250d admin admin] PATH_INFO: `/v3/projects/67deafb7b6f7467ca6739b62006df96c/users/051e1d2df6c940d083da6f7707dc2231/roles/454bd427dfc04030a4ec2b0e8a1ed237` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:23:16.427203 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f9d7d-68a5-4506-9b00-14513999250d admin admin] RBAC: Authorizing `identity:create_grant(project_id=67deafb7b6f7467ca6739b62006df96c, user_id=051e1d2df6c940d083da6f7707dc2231, role_id=454bd427dfc04030a4ec2b0e8a1ed237)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:23:16.429840 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f9d7d-68a5-4506-9b00-14513999250d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:23:16.544642 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1472/2959] 10.222.0.59 () {60 vars in 1483 bytes} [Mon Aug 29 23:23:16 2022] PUT /identity/v3/projects/67deafb7b6f7467ca6739b62006df96c/users/051e1d2df6c940d083da6f7707dc2231/roles/454bd427dfc04030a4ec2b0e8a1ed237 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:23:16.554901 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a919ce-6ffb-48b6-9593-b5dc4dd99c95 None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:23:16.555185 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a919ce-6ffb-48b6-9593-b5dc4dd99c95 None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:23:16.555458 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a919ce-6ffb-48b6-9593-b5dc4dd99c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:23:16.584604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-91a919ce-6ffb-48b6-9593-b5dc4dd99c95 None None] MFA Rules not processed for user `051e1d2df6c940d083da6f7707dc2231`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:23:16.585807 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-91a919ce-6ffb-48b6-9593-b5dc4dd99c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:23:16.650118 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1488/2960] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:23:16 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:23:16.661065 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f9d7d-68a5-4506-9b00-14513999250d admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:23:16.663050 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-190f9d7d-68a5-4506-9b00-14513999250d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:23:16.679905 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:23:16.725900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f9d7d-68a5-4506-9b00-14513999250d admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:23:16.729716 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786af4e-9ea9-45ab-9e7c-839b319a63a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92af56c889e640e7815c1641c05a9af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:23:16.730329 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786af4e-9ea9-45ab-9e7c-839b319a63a6 service cinder] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:23:16.730612 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786af4e-9ea9-45ab-9e7c-839b319a63a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:23:16.730880 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786af4e-9ea9-45ab-9e7c-839b319a63a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:23:16.740570 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786af4e-9ea9-45ab-9e7c-839b319a63a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:23:16.742389 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786af4e-9ea9-45ab-9e7c-839b319a63a6 service cinder] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:23:16.751931 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1473/2961] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:23:16 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:23:26.646867 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:23:26.818831 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=20 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:23:27.723951 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a919ce-6ffb-48b6-9593-b5dc4dd99c95 None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:23:27.725342 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-91a919ce-6ffb-48b6-9593-b5dc4dd99c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:23:27.860421 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a919ce-6ffb-48b6-9593-b5dc4dd99c95 None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:23:27.870801 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93efad-f4e1-48ab-a17b-0f97bcd7eb2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:23:27.870801 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93efad-f4e1-48ab-a17b-0f97bcd7eb2c service placement] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:23:27.870801 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93efad-f4e1-48ab-a17b-0f97bcd7eb2c service placement] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:23:27.870801 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93efad-f4e1-48ab-a17b-0f97bcd7eb2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:23:27.871519 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-4b93efad-f4e1-48ab-a17b-0f97bcd7eb2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:23:27.887103 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93efad-f4e1-48ab-a17b-0f97bcd7eb2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:23:27.887103 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93efad-f4e1-48ab-a17b-0f97bcd7eb2c service placement] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:23:27.894945 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1489/2962] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:23:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:23:37.408424 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786af4e-9ea9-45ab-9e7c-839b319a63a6 service cinder] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:23:37.485117 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786af4e-9ea9-45ab-9e7c-839b319a63a6 service cinder] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:23:37.490283 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06421e-350b-43a2-842b-0f6684a53381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:23:37.490900 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06421e-350b-43a2-842b-0f6684a53381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:23:37.491207 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06421e-350b-43a2-842b-0f6684a53381 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:23:37.491479 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06421e-350b-43a2-842b-0f6684a53381 admin admin] PATH_INFO: `/v3/users/051e1d2df6c940d083da6f7707dc2231` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:23:37.492683 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a06421e-350b-43a2-842b-0f6684a53381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=051e1d2df6c940d083da6f7707dc2231)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:23:37.494614 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a06421e-350b-43a2-842b-0f6684a53381 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:23:37.638474 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5a06421e-350b-43a2-842b-0f6684a53381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '051e1d2df6c940d083da6f7707dc2231'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:23:37.645623 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5a06421e-350b-43a2-842b-0f6684a53381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '051e1d2df6c940d083da6f7707dc2231'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:23:37.646296 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5a06421e-350b-43a2-842b-0f6684a53381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '051e1d2df6c940d083da6f7707dc2231'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:23:37.658957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5a06421e-350b-43a2-842b-0f6684a53381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '051e1d2df6c940d083da6f7707dc2231'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:23:37.659628 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-5a06421e-350b-43a2-842b-0f6684a53381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '051e1d2df6c940d083da6f7707dc2231'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:23:37.756271 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1474/2963] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:23:37 2022] DELETE /identity/v3/users/051e1d2df6c940d083da6f7707dc2231 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:23:37.760834 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccb705-eb8d-464d-9321-545e8c4aee7a None None] REQUEST_METHOD: `POST` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:23:37.761304 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccb705-eb8d-464d-9321-545e8c4aee7a None None] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:23:37.761402 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccb705-eb8d-464d-9321-545e8c4aee7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:23:37.780129 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.auth.core [None req-1bccb705-eb8d-464d-9321-545e8c4aee7a None None] MFA Rules not processed for user `b7cd207626714450952ac5fa63cc9cad`. Rule list: `[]` (Enabled: `True`). {{(pid=82165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 23:23:37.781337 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-1bccb705-eb8d-464d-9321-545e8c4aee7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:23:37.839179 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1490/2964] 10.222.0.59 () {58 vars in 958 bytes} [Mon Aug 29 23:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 23:23:37.849608 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06421e-350b-43a2-842b-0f6684a53381 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:23:37.851028 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-5a06421e-350b-43a2-842b-0f6684a53381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:23:37.909397 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06421e-350b-43a2-842b-0f6684a53381 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:23:37.913254 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b9033-214f-458d-a3b3-64e82b716fcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:23:37.913881 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b9033-214f-458d-a3b3-64e82b716fcb service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:23:37.914159 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b9033-214f-458d-a3b3-64e82b716fcb service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:23:37.914509 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b9033-214f-458d-a3b3-64e82b716fcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:23:37.924340 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b9033-214f-458d-a3b3-64e82b716fcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:23:37.926021 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b9033-214f-458d-a3b3-64e82b716fcb service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:23:37.935661 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1475/2965] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:23:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:23:38.315499 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bccb705-eb8d-464d-9321-545e8c4aee7a None None] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:23:38.317136 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-1bccb705-eb8d-464d-9321-545e8c4aee7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:23:38.332621 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bccb705-eb8d-464d-9321-545e8c4aee7a None None] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:23:38.337656 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ce45e-accd-4c0b-86f1-cdac88c89e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:23:38.340494 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ce45e-accd-4c0b-86f1-cdac88c89e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:23:38.340494 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ce45e-accd-4c0b-86f1-cdac88c89e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:23:38.340494 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ce45e-accd-4c0b-86f1-cdac88c89e11 admin admin] PATH_INFO: `/v3/projects/67deafb7b6f7467ca6739b62006df96c` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:23:38.340494 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ce45e-accd-4c0b-86f1-cdac88c89e11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67deafb7b6f7467ca6739b62006df96c)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:23:38.341996 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ce45e-accd-4c0b-86f1-cdac88c89e11 admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:23:38.430329 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:23:38.445608 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-992ce45e-accd-4c0b-86f1-cdac88c89e11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67deafb7b6f7467ca6739b62006df96c', 'request_id': 'req-992ce45e-accd-4c0b-86f1-cdac88c89e11'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:23:38.451669 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-992ce45e-accd-4c0b-86f1-cdac88c89e11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67deafb7b6f7467ca6739b62006df96c', 'request_id': 'req-992ce45e-accd-4c0b-86f1-cdac88c89e11'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:23:38.457782 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-992ce45e-accd-4c0b-86f1-cdac88c89e11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67deafb7b6f7467ca6739b62006df96c', 'request_id': 'req-992ce45e-accd-4c0b-86f1-cdac88c89e11'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:23:38.536161 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-992ce45e-accd-4c0b-86f1-cdac88c89e11 admin admin] The token cache is being invalidate because project 67deafb7b6f7467ca6739b62006df96c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:23:38.536557 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-992ce45e-accd-4c0b-86f1-cdac88c89e11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:23:38.537173 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-992ce45e-accd-4c0b-86f1-cdac88c89e11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:23:38.538891 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1491/2966] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:23:38 2022] DELETE /identity/v3/projects/67deafb7b6f7467ca6739b62006df96c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:23:47.931320 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:23:48.542381 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=14,keystone:DELETE=5,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:24:14.437257 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b9033-214f-458d-a3b3-64e82b716fcb service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:24:14.438650 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-692b9033-214f-458d-a3b3-64e82b716fcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:24:14.450494 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b9033-214f-458d-a3b3-64e82b716fcb service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:24:14.498210 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f969a-6ab4-4706-8b34-81fe62cd898a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:24:14.498807 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f969a-6ab4-4706-8b34-81fe62cd898a service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:24:14.499075 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f969a-6ab4-4706-8b34-81fe62cd898a service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:24:14.499352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f969a-6ab4-4706-8b34-81fe62cd898a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:24:14.501048 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-ad1f969a-6ab4-4706-8b34-81fe62cd898a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:24:14.570257 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f969a-6ab4-4706-8b34-81fe62cd898a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:24:14.571678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f969a-6ab4-4706-8b34-81fe62cd898a service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:24:14.583532 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1476/2967] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:24:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:24:15.765274 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ce45e-accd-4c0b-86f1-cdac88c89e11 admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:24:15.774750 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-992ce45e-accd-4c0b-86f1-cdac88c89e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:24:15.779341 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ce45e-accd-4c0b-86f1-cdac88c89e11 admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:24:15.783716 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0a1ee-21db-4218-9037-0ab794a3fe27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:24:15.784693 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0a1ee-21db-4218-9037-0ab794a3fe27 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:24:15.784972 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0a1ee-21db-4218-9037-0ab794a3fe27 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:24:15.785248 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0a1ee-21db-4218-9037-0ab794a3fe27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:24:15.786926 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-d4c0a1ee-21db-4218-9037-0ab794a3fe27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:24:15.801760 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0a1ee-21db-4218-9037-0ab794a3fe27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:24:15.805758 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0a1ee-21db-4218-9037-0ab794a3fe27 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:24:15.814297 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1492/2968] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:24:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:24:24.577950 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=54 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:24:25.809268 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=6 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:24:29.260102 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f969a-6ab4-4706-8b34-81fe62cd898a service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:24:29.277872 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f969a-6ab4-4706-8b34-81fe62cd898a service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:24:29.281579 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6bcd5-803e-4836-97cb-07b65607b3ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:24:29.282177 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6bcd5-803e-4836-97cb-07b65607b3ec service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:24:29.282449 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6bcd5-803e-4836-97cb-07b65607b3ec service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:24:29.282708 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6bcd5-803e-4836-97cb-07b65607b3ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:24:29.288262 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-74f6bcd5-803e-4836-97cb-07b65607b3ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:24:29.298980 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f6bcd5-803e-4836-97cb-07b65607b3ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:24:29.300482 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f6bcd5-803e-4836-97cb-07b65607b3ec service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:24:29.308782 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1477/2969] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:24:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:24:29.395416 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0a1ee-21db-4218-9037-0ab794a3fe27 service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:24:29.396838 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-d4c0a1ee-21db-4218-9037-0ab794a3fe27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:24:29.407345 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0a1ee-21db-4218-9037-0ab794a3fe27 service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:24:29.411054 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6eb021-51fb-40ba-a248-7849b47fe2e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:24:29.411642 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6eb021-51fb-40ba-a248-7849b47fe2e1 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:24:29.411915 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6eb021-51fb-40ba-a248-7849b47fe2e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:24:29.412208 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6eb021-51fb-40ba-a248-7849b47fe2e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:24:29.414473 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-2c6eb021-51fb-40ba-a248-7849b47fe2e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:24:29.428640 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6eb021-51fb-40ba-a248-7849b47fe2e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:24:29.430075 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6eb021-51fb-40ba-a248-7849b47fe2e1 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:24:29.441898 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1493/2970] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:24:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:24:39.305437 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=6 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:24:39.436221 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=6 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:25:36.444972 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6bcd5-803e-4836-97cb-07b65607b3ec service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:25:36.474399 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6bcd5-803e-4836-97cb-07b65607b3ec service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:25:36.478064 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ecdf4-dbd5-44af-afad-27b87ad5bc93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:25:36.478656 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ecdf4-dbd5-44af-afad-27b87ad5bc93 service neutron] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:25:36.478994 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ecdf4-dbd5-44af-afad-27b87ad5bc93 service neutron] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:25:36.479322 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ecdf4-dbd5-44af-afad-27b87ad5bc93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:25:36.481038 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6b9ecdf4-dbd5-44af-afad-27b87ad5bc93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:25:36.490542 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ecdf4-dbd5-44af-afad-27b87ad5bc93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:25:36.491990 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ecdf4-dbd5-44af-afad-27b87ad5bc93 service neutron] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:25:36.500678 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1478/2971] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:25:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:25:46.497003 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=15 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:26:04.100360 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6eb021-51fb-40ba-a248-7849b47fe2e1 service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:26:04.101933 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-2c6eb021-51fb-40ba-a248-7849b47fe2e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:26:04.130049 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6eb021-51fb-40ba-a248-7849b47fe2e1 service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:26:04.136091 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ff5ab-e2c5-4c2b-a7de-3f6a883f0071 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:26:04.136765 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ff5ab-e2c5-4c2b-a7de-3f6a883f0071 service nova] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:26:04.137003 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ff5ab-e2c5-4c2b-a7de-3f6a883f0071 service nova] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:26:04.137279 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ff5ab-e2c5-4c2b-a7de-3f6a883f0071 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:26:04.139261 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-d90ff5ab-e2c5-4c2b-a7de-3f6a883f0071 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:26:04.154835 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ff5ab-e2c5-4c2b-a7de-3f6a883f0071 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:26:04.155476 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ff5ab-e2c5-4c2b-a7de-3f6a883f0071 service nova] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:26:04.168001 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1494/2972] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:26:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:26:06.853587 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ecdf4-dbd5-44af-afad-27b87ad5bc93 service neutron] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:26:06.855584 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-6b9ecdf4-dbd5-44af-afad-27b87ad5bc93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:26:06.873078 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ecdf4-dbd5-44af-afad-27b87ad5bc93 service neutron] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:26:06.879518 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a6c48-6663-476a-a588-bda33b9c83f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f21105679f245a19ea4850e6c010dbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:26:06.880352 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a6c48-6663-476a-a588-bda33b9c83f9 service nova] REQUEST_METHOD: `GET` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:26:06.880637 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a6c48-6663-476a-a588-bda33b9c83f9 service nova] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:26:06.880912 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a6c48-6663-476a-a588-bda33b9c83f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:26:06.882703 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3b9a6c48-6663-476a-a588-bda33b9c83f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:26:06.896230 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a6c48-6663-476a-a588-bda33b9c83f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:26:06.897809 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a6c48-6663-476a-a588-bda33b9c83f9 service nova] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:26:06.909884 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1479/2973] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:26:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 23:26:12.672081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ff5ab-e2c5-4c2b-a7de-3f6a883f0071 service nova] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:26:12.672081 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-d90ff5ab-e2c5-4c2b-a7de-3f6a883f0071 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:26:12.741647 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ff5ab-e2c5-4c2b-a7de-3f6a883f0071 service nova] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:26:12.746772 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be94503-d393-4822-a50e-0d399a168cb2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:26:12.747367 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be94503-d393-4822-a50e-0d399a168cb2 service placement] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:26:12.747639 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be94503-d393-4822-a50e-0d399a168cb2 service placement] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:26:12.747906 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be94503-d393-4822-a50e-0d399a168cb2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:26:12.749622 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-8be94503-d393-4822-a50e-0d399a168cb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:26:12.760930 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be94503-d393-4822-a50e-0d399a168cb2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:26:12.762476 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be94503-d393-4822-a50e-0d399a168cb2 service placement] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:26:12.772894 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1495/2974] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:26:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:26:16.820332 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a6c48-6663-476a-a588-bda33b9c83f9 service nova] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:26:16.821892 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-3b9a6c48-6663-476a-a588-bda33b9c83f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:26:16.889253 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a6c48-6663-476a-a588-bda33b9c83f9 service nova] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:26:16.894637 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d25815-3b4c-4c25-a4dc-cc958a3797a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:26:16.895255 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d25815-3b4c-4c25-a4dc-cc958a3797a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:26:16.895533 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d25815-3b4c-4c25-a4dc-cc958a3797a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:26:16.895805 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d25815-3b4c-4c25-a4dc-cc958a3797a1 admin admin] PATH_INFO: `/v3/users/d3acbdab00c64d84871545062a5b5311` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:26:16.896945 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d25815-3b4c-4c25-a4dc-cc958a3797a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3acbdab00c64d84871545062a5b5311)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:26:16.898929 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d25815-3b4c-4c25-a4dc-cc958a3797a1 admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:26:17.073960 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-a5d25815-3b4c-4c25-a4dc-cc958a3797a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3acbdab00c64d84871545062a5b5311'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:26:17.080889 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-a5d25815-3b4c-4c25-a4dc-cc958a3797a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3acbdab00c64d84871545062a5b5311'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:26:17.081562 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-a5d25815-3b4c-4c25-a4dc-cc958a3797a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3acbdab00c64d84871545062a5b5311'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:26:17.093244 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-a5d25815-3b4c-4c25-a4dc-cc958a3797a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3acbdab00c64d84871545062a5b5311'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:26:17.093988 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-a5d25815-3b4c-4c25-a4dc-cc958a3797a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3acbdab00c64d84871545062a5b5311'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:26:17.162380 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1480/2975] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:26:16 2022] DELETE /identity/v3/users/d3acbdab00c64d84871545062a5b5311 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:26:17.405844 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be94503-d393-4822-a50e-0d399a168cb2 service placement] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:26:17.419294 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be94503-d393-4822-a50e-0d399a168cb2 service placement] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:26:17.479614 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d8ec0-b67f-4552-8af6-b02d891ca86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:26:17.480280 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d8ec0-b67f-4552-8af6-b02d891ca86e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:26:17.480559 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d8ec0-b67f-4552-8af6-b02d891ca86e admin admin] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:26:17.480829 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d8ec0-b67f-4552-8af6-b02d891ca86e admin admin] PATH_INFO: `/v3/projects/fc6b1df0e5104fd6abb5dcf5497d2cb1` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:26:17.481962 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d8ec0-b67f-4552-8af6-b02d891ca86e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc6b1df0e5104fd6abb5dcf5497d2cb1)` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:26:17.483865 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d8ec0-b67f-4552-8af6-b02d891ca86e admin admin] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:26:17.575576 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7a0d8ec0-b67f-4552-8af6-b02d891ca86e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc6b1df0e5104fd6abb5dcf5497d2cb1', 'request_id': 'req-7a0d8ec0-b67f-4552-8af6-b02d891ca86e'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:26:17.581839 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7a0d8ec0-b67f-4552-8af6-b02d891ca86e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc6b1df0e5104fd6abb5dcf5497d2cb1', 'request_id': 'req-7a0d8ec0-b67f-4552-8af6-b02d891ca86e'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:26:17.587436 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7a0d8ec0-b67f-4552-8af6-b02d891ca86e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc6b1df0e5104fd6abb5dcf5497d2cb1', 'request_id': 'req-7a0d8ec0-b67f-4552-8af6-b02d891ca86e'} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:26:17.590993 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:26:17.654851 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7a0d8ec0-b67f-4552-8af6-b02d891ca86e admin admin] The token cache is being invalidate because project fc6b1df0e5104fd6abb5dcf5497d2cb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:26:17.655104 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7a0d8ec0-b67f-4552-8af6-b02d891ca86e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:26:17.655749 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.notifications [None req-7a0d8ec0-b67f-4552-8af6-b02d891ca86e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:26:17.657887 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1496/2976] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:26:17 2022] DELETE /identity/v3/projects/fc6b1df0e5104fd6abb5dcf5497d2cb1 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:26:17.661957 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d25815-3b4c-4c25-a4dc-cc958a3797a1 admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:26:17.663360 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-a5d25815-3b4c-4c25-a4dc-cc958a3797a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:26:17.677799 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d25815-3b4c-4c25-a4dc-cc958a3797a1 admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:26:17.734450 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eba64a-1079-41fd-b14a-257e7ad3c82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:26:17.735109 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eba64a-1079-41fd-b14a-257e7ad3c82d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:26:17.735379 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eba64a-1079-41fd-b14a-257e7ad3c82d admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:26:17.735646 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eba64a-1079-41fd-b14a-257e7ad3c82d admin admin] PATH_INFO: `/v3/users/1d0c4f3cfb594a13821d1082b07cbbc4` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:26:17.736797 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eba64a-1079-41fd-b14a-257e7ad3c82d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d0c4f3cfb594a13821d1082b07cbbc4)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:26:17.738762 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eba64a-1079-41fd-b14a-257e7ad3c82d admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:26:17.744105 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:26:17.925003 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-84eba64a-1079-41fd-b14a-257e7ad3c82d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d0c4f3cfb594a13821d1082b07cbbc4'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:26:17.931931 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-84eba64a-1079-41fd-b14a-257e7ad3c82d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d0c4f3cfb594a13821d1082b07cbbc4'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:26:17.932637 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-84eba64a-1079-41fd-b14a-257e7ad3c82d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d0c4f3cfb594a13821d1082b07cbbc4'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:26:17.944971 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-84eba64a-1079-41fd-b14a-257e7ad3c82d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d0c4f3cfb594a13821d1082b07cbbc4'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:26:17.956111 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-84eba64a-1079-41fd-b14a-257e7ad3c82d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d0c4f3cfb594a13821d1082b07cbbc4'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:26:18.050045 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1481/2977] 10.222.0.59 () {58 vars in 1224 bytes} [Mon Aug 29 23:26:17 2022] DELETE /identity/v3/users/1d0c4f3cfb594a13821d1082b07cbbc4 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 23:26:18.059525 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d8ec0-b67f-4552-8af6-b02d891ca86e admin admin] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:26:18.060989 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-7a0d8ec0-b67f-4552-8af6-b02d891ca86e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:26:18.118981 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d8ec0-b67f-4552-8af6-b02d891ca86e admin admin] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:26:18.167070 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae301d0-00c0-49d4-ba6d-b026b0d53e08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb802d01e71b486cb40c6388430d674d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:26:18.167677 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae301d0-00c0-49d4-ba6d-b026b0d53e08 service neutron] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:26:18.167952 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae301d0-00c0-49d4-ba6d-b026b0d53e08 service neutron] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:26:18.168253 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae301d0-00c0-49d4-ba6d-b026b0d53e08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:26:18.170000 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-4ae301d0-00c0-49d4-ba6d-b026b0d53e08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:26:18.231060 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae301d0-00c0-49d4-ba6d-b026b0d53e08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:26:18.232439 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae301d0-00c0-49d4-ba6d-b026b0d53e08 service neutron] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:26:18.242604 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1497/2978] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:26:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:26:18.503204 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eba64a-1079-41fd-b14a-257e7ad3c82d admin admin] Authenticating user token {{(pid=82166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:26:18.504698 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.fernet_utils [None req-84eba64a-1079-41fd-b14a-257e7ad3c82d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:26:18.520564 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eba64a-1079-41fd-b14a-257e7ad3c82d admin admin] Validating token access rules against request {{(pid=82166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:26:18.525320 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98dbbbe-388f-46df-9204-5b357be20e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7cd207626714450952ac5fa63cc9cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3d78b58a69d4b10868c64742da904fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:26:18.525956 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98dbbbe-388f-46df-9204-5b357be20e5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:26:18.526233 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98dbbbe-388f-46df-9204-5b357be20e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:26:18.526526 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98dbbbe-388f-46df-9204-5b357be20e5c admin admin] PATH_INFO: `/v3/projects/4af31fd1c14e4db18aee028aebe3edd1` {{(pid=82166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:26:18.527634 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98dbbbe-388f-46df-9204-5b357be20e5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4af31fd1c14e4db18aee028aebe3edd1)` {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:26:18.530906 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98dbbbe-388f-46df-9204-5b357be20e5c admin admin] RBAC: Authorization granted {{(pid=82166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:26:18.639126 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c98dbbbe-388f-46df-9204-5b357be20e5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4af31fd1c14e4db18aee028aebe3edd1', 'request_id': 'req-c98dbbbe-388f-46df-9204-5b357be20e5c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:26:18.728480 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c98dbbbe-388f-46df-9204-5b357be20e5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4af31fd1c14e4db18aee028aebe3edd1', 'request_id': 'req-c98dbbbe-388f-46df-9204-5b357be20e5c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:26:18.734340 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c98dbbbe-388f-46df-9204-5b357be20e5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4af31fd1c14e4db18aee028aebe3edd1', 'request_id': 'req-c98dbbbe-388f-46df-9204-5b357be20e5c'} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:26:18.745448 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c98dbbbe-388f-46df-9204-5b357be20e5c admin admin] The token cache is being invalidate because project 4af31fd1c14e4db18aee028aebe3edd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 23:26:18.745849 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c98dbbbe-388f-46df-9204-5b357be20e5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:26:18.746467 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG keystone.notifications [None req-c98dbbbe-388f-46df-9204-5b357be20e5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 23:26:18.747944 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: [pid: 82166|app: 0|req: 1482/2979] 10.222.0.59 () {58 vars in 1233 bytes} [Mon Aug 29 23:26:18 2022] DELETE /identity/v3/projects/4af31fd1c14e4db18aee028aebe3edd1 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 23:26:28.238309 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=72 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:26:28.742453 dv-cinder-854268-3-iscsi devstack@keystone.service[82166]: DEBUG dbcounter [-] [82166] Writing DB stats keystone:SELECT=50,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 23:28:30.700242 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae301d0-00c0-49d4-ba6d-b026b0d53e08 service neutron] Authenticating user token {{(pid=82165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 23:28:30.701636 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-4ae301d0-00c0-49d4-ba6d-b026b0d53e08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:28:30.772432 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae301d0-00c0-49d4-ba6d-b026b0d53e08 service neutron] Validating token access rules against request {{(pid=82165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 23:28:30.836698 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588f7bd-4864-4a29-b3e3-4db328e4b170 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7215ee41033742d79671f4f42b81cd06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7799820fc7c64fd6a99a0a9ccf1338e3', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 23:28:30.837362 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588f7bd-4864-4a29-b3e3-4db328e4b170 service placement] REQUEST_METHOD: `GET` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 23:28:30.837634 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588f7bd-4864-4a29-b3e3-4db328e4b170 service placement] SCRIPT_NAME: `/identity` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 23:28:30.837998 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588f7bd-4864-4a29-b3e3-4db328e4b170 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 23:28:30.839753 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.fernet_utils [None req-6588f7bd-4864-4a29-b3e3-4db328e4b170 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 23:28:30.851328 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588f7bd-4864-4a29-b3e3-4db328e4b170 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 23:28:30.853199 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588f7bd-4864-4a29-b3e3-4db328e4b170 service placement] RBAC: Authorization granted {{(pid=82165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 23:28:30.863163 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: [pid: 82165|app: 0|req: 1498/2980] 10.222.0.59 () {60 vars in 1365 bytes} [Mon Aug 29 23:28:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 23:28:40.859061 dv-cinder-854268-3-iscsi devstack@keystone.service[82165]: DEBUG dbcounter [-] [82165] Writing DB stats keystone:SELECT=58 {{(pid=82165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}